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Abstract  of  Dissertation  Presented  to  the  Graduate  School 
of  the  University  of  Rorida  in  Partial  Fulfillment  of  the 
Requirements  for  the  Degree  of  Doctor  of  Philosophy 

THE  EFFECTS  OF  PROACTIVE  DIGITAL  VIDEO  GUIDE 
ASSISTANCE,  LOCUS  OF  CONTROL,  AND  COMPUTER 
EXPERIENCE  ON  SUCCESS  IN  A  COMPUTERIZED 
PROBLEM-SOLVING  ACTIVITY 

By 

Sebastian  Foti 
April  1994 

Chairman:  Elroy  J.  Bolduc 

Major  Department:  Instruction  and  Curriculum 

This  purpose  of  this  study  was  to  determine  if  there  are  differences  in 
problem-solving  outcomes  between  experienced  and  nonexperienced 
learners  with  varying  locus-of-control  orientations  due  to  the  effect  of 
varied  types  of  digitized  video  guide  assistance  available  in  a  problem 
space.  In  addition,  the  study  examines  the  perceptions  of  the  subjects 
related  to  types  of  digitized  video  guide-based  help  intervention,  problem 
difficulty  level,  and  usefulness  of  help  facilities. 

Data  were  collected  on  105  subjects,  juniors  and  seniors  at  the 
University  of  Florida.  Each  subject  responded  to  the  Nowicki-Strickland 
Locus  of  Control  Inventory,  a  one-page  computer  experience  survey,  and  a 
computerized  problem-solving  activity.  The  problem-solving  aspect  of  the 
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treatment  also  contained  eight  attitudinal  questions  which  incorporated  a 
semantic  differential  scale.  During  the  computerized  problem-solving 
activity,  either  proactive  or  request-only  help  was  given  using  digitized 
videos  of  male,  bow-tie-wearing  guide  "assistant." 

Logistic  regression,  multiple  regression,  and  ANOVA  were  used  to 
analyze  the  data.  The  results  of  the  analyses  (p<  .05)  revealed  that  offering 
proactive  digital  video  guide  help  increases  the  likelihood  that  users  will 
request  help  on  their  own,  and  users  who  do  request  such  help  are  more 
likely  to  solve  the  problem  correctly.  Although  results  of  the  study 
indicate  that  locus  of  control  and  experience  do  not  impact  the  likelihood  of 
success  with  the  problem-solving  activity,  locus-of-control  orientation  was 
found  to  be  a  significant  predictor  of  whether  the  user  will  fmd  digital 
video  guide  help  useful  and  of  whether  the  user  will  enjoy  the  problem- 
solving  activity.  Moreover,  the  more  externally  oriented  the  user,  the 
more  likely  they  will  be  to  (a)  find  digital  video  guide  help  useful  and  (b) 
enjoy  the  activity  itself. 

The  results  of  this  study  indicate  that  digital  video  guide  assistance  can 
be  effective  with  beginners  in  a  problem-solving  activity.  Because  locus  of 
control  was  found  to  be  a  significant  predictor  of  certain  user  attitudes,  its 
relationship,  as  well  as  the  relationship  of  other  social  learning  theory 
constructs  to  the  man-machine  interface  merits  further  research. 
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CHAPTER  1 
INTRODUCTION 


Recent  advances  in  computer  technology  and  behavioral  science  have 
transformed  the  once  educationally  primitive  domain  of  machine-mediated 
learning  into  a  robust,  vital  area  of  research  in  which  educators,  cognitive 
psychologists,  and  computer  engineers  share  visions  of  the  future.  Beyond 
using  computer  technology  to  present  instructional  materials  using 
traditional  deUvery  modes  such  as  drill  and  practice,  the  linear  tutorial,  and 
question-answer  interactions,  today's  system  designers  are  applying  human 
learning  principles  to  computer  systems  at  every  level  of  operation. 
Brenda  Laurel  (1990)  notes. 

Computers  behave.  Computational  tools  and  applications  can 
be  said  to  have  predispositions  to  behave  in  certain  ways  on 
both  functional  and  stylistic  levels.  Interfaces  are  designed  to 
communicate  those  predispositions  to  users,  thereby  enabling 
them  to  understand,  predict  the  results  of,  and  successfully 
deploy  the  associated  behaviors,  (p.  355) 

Since  communicating  dispositions  has  been  essentially  a  human  task,  it 
is  natural  that  the  metaphors  used  by  some  interface  designers  have  been 
based  on  living  organisms.  This  anthropomorphic  view  of  the  machine  has 
yielded  interfaces  incorporating  semi-intelligent  animated  "characters"  that 
appear  to  perform  many  routine  tasks  for  the  computer  user.  These 
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characters  represent  a  system  which  may  supply  expertise,  skill,  and  labor 
in  response  to  the  user's  needs.  As  Laurel  notes,  "All  of  the  computer- 
based  personae  that  weave  through  popular  culture  have  one  thing  in 
common:  they  mediate  a  relationship  between  the  labyrinthine  precision  of 
computers  and  the  fuzzy  complexity  of  man"  (p.  355).  The  form  that  these 
characters  take  has  an  effect  on  the  user's  perceptions  of  the  system.  For 
example,  Apple  Computer®  Inc.  has  provided  a  variety  of  interface 
characters  for  the  speech  recognition  software  on  Macintosh®  AV 
computer  systems.  Some  of  these  characters  are  human-like,  and  some  are 
simple  machine-like  indicators  similar  to  those  found  on  a  stereo  amplifier. 
Although  the  functions  fulfilled  by  these  interface  artifacts  are  the  same, 
the  metaphors  used  are  very  different.  Functionally,  these  characters  are 
designed  to  "guide"  the  user  through  the  task  at  hand  or  act  as  "agents"  to 
do  specific  tasks  for  the  user.  Although  the  computer  industry  has  high 
hopes  for  systems  which  provide  such  assistance,  very  little  research  has 
been  done  to  determine  which  pedagogical  techniques  are  most  effective 
under  which  pedagogical  conditions. 

This  study  uses  a  video  character  as  a  guide  that  assists  the  user  in  a 
problem-solving  activity.  The  "video  guide"  in  the  study  is  an  actual 
person  who  was  filmed  in  a  recording  studio.  The  video  guide  offers  help 
during  the  activity  in  one  of  two  ways:  either  proactively,  that  is, 
automatically  without  being  asked,  or  on-demand,  when  the  user  requests 
it.  The  video  guide  is  displayed  in  a  "Help  Box"  which  is  an  area  in  the 
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lower  left  comer  of  the  activity  space  (see  Appendix  A).  While  the  user  is 
becoming  familiar  with  the  system,  Help  is  context  sensitive,  that  is,  the 
assistance  given  is  appropriate  for  user's  situation. 

The  paucity  of  research  in  the  field  of  Human  Computer  Interaction 
(HCI)  is  due  in  part  to  problems  associated  with  conducting  empirical 
research  in  the  field.  Early  research  attempted  to  apply  strictly  the 
scientific  method  in  laboratory  studies.  According  to  John  Carrol  (1991) 
of  the  IBM  Thomas  J.  Watson  Research  Center, 

The  scientific  method  implies  dichotomous  contrasts,  and  this 
works  well  only  if  the  required  controls  are  possible  and 
appropriate.  In  HCI  these  requirements  are  generally  not 
met.  ...  In  HCI,  the  real  world  consists  of  various  artifacts 
(software,  hardware,  documentation)  embedded  in  contexts  of 
work  and  social  interaction.  In  constructing  a  science  of  HCI, 
we  make  an  intermediary  representation  of  this  reality,  usually 
involving  selection  and  simplification.  Barnard  calls  this  a 
"discovery"  representation,  (p.  4) 

Barnard  (1991)  believes  strongly  in  the  use  of  discovery 
representations  and  promotes  the  use  of  such  interaction  scenarios  to  assess 
how  well  a  given  HCI  theory  meets  the  actual  needs  found  in  the 
application  domain. 

The  discovery  representation  used  in  this  study  incorporated  an 
interaction  scenario  in  which  the  user  is  involved  in  a  computer-based  logic 
problem-solving  activity.  This  discovery  representation  will  be  referred  to 
as  the  problem  space.  The  study  also  incorporated  the  use  of  an  animated 
digitized  video  character  which  will  be  referred  to  as  a  guide.  The  guide 
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was  used  primarily  in  an  advisory  capacity,  offering  context-sensitive 
assistance. 

Since  Newell  and  Simon  (1972)  focused  on  the  man-machine 
dialogue  as  a  manifestation  of  a  physical  symbol  system,  research  in  the 
area  of  computerized  advice-giving  has  been  expanding  rapidly  (Carroll  & 
McKendree,  1987).  The  use  of  a  guide  creates  the  illusion  of  a  physical 
source  for  the  advice.  To  some  extent,  it  is  one  level  of  abstraction  away 
from  the  programmer-user  interaction  found  in  traditional  text-based 
systems. 

Statement  of  the  Problem 
This  study  applied  theories  found  in  education  and  cognitive 
psychology  to  the  application  of  guide-based  help  in  interaction  scenarios 
found  in  the  growing  field  of  Human  Computer  Interaction  (HCI). 
The  study  investigated  three  broad  questions: 

1.  Does  digital  video  guide-based  help  intervention  support  a  user's 
efforts  in  a  problem  space? 

2.  Is  the  effectiveness  of  proactive  digital  video  guide-based  help 
affected  by  parameters  such  as  locus  of  control  and  computer  experience? 

3.  What  factors  contribute  to  the  user's  perception  of  the  video 
guide-based  help  system? 

Specifically,  the  study  was  designed  to  determine  whether  proactive, 
(system-initiated,  context-sensitive)  digital  video  guide-based  help 
intervention  can  be  more  effective  than  on-demand  (user-initiated. 
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context-sensitive)  help  intervention  alone.  In  short,  the  control  of  access  to 
help  could  come  from  two  sources,  either  the  computer  'decides'  or  the 
user  decides.  In  addition,  factors  that  contribute  to  the  user's  perception  of 
the  digital  video  guide-based  help  features  were  examined. 

The  subjects  in  this  study  were  undergraduate  students  at  the  University 
of  Florida  who  were  enrolled  in  introductory  Instructional  Computing 
classes  who  took  a  locus  of  control  measure.  Locus  of  control  measures 
reflect  the  extent  to  which  control  over  valued  reinforcements  is  perceived 
as  internally  or  externally  oriented  (Rotter,  1966).  The  treatments 
presented  mediated  learning  experiences  alternatively  offered  with 
on-demand  guide  assistance  and  automatic,  proactive,  system-initiated  guide 
assistance  to  participants  actively  involved  in  the  problem  space. 

Need  for  the  Study 

Incorporating  a  human  character  in  the  human-computer  interface  adds 
several  dimensions  to  an  advisory  system.  In  addition  to  the  media 
considerations  that  audio  and  video  capabiHties  furnish,  questions 
concerning  the  anthropomorphic  qualities  of  the  intellectual  assistant 
become  central.  The  'role'  of  the  computer  character  is  also  in  question. 
Is  the  character  viewed  as  being  in  some  way  in  control  of  the  interaction? 
The  complexity  of  advisory  systems  is  reflected  in  Jackson  and  Lefrere's 
(1984)  statements: 

Advising  is  more  than  describing  or  demonstrating  because  it 
suggests  both  the  existence  of  alternatives  and  the  possibility  of 
a  preference  on  someone's  part.  Advising  also  suggests  the 


existence  of  a  problem,  which  may  involve  more  than  the 
mere  lack  of  information,  (p.  65) 

On  the  other  hand,  the  human  character's  more  natural  conversational 
approach  to  advice-giving  seems  superior  to  alternatives  in  other  ways. 
Most  current  user  interfaces  employ  a  graphical  user  interface  (GUI)  to 
present  information  which  may  then  be  manipulated  using  iconic  or  textual 
tools.  These  direct  manipulation  interfaces,  although  powerful,  present  the 
user  with  "too  much  to  do  or  see.  Auditory  I/O  is  a  natural,  available,  and 
systematically  underutilized  channel  for  enhancing  user-computer 
communication"  (Mountford  &  Gaver,  1990,  p.  320). 

The  concept  of  what  is  "natural"  in  an  interface  has  been  the  topic  of 
several  speculative  articles  in  the  literature.  Norman  (1993)  states, 
"Naturalness  is  actually  specified  as  the  complexity  of  the  description  of  the 
mapping  between  the  representation  and  the  thing  being  represented"  (p. 
261).  An  informative  summary  of  the  anthropomorphic  approaches  to 
interface  design  was  given  in  Eberts  and  Eberts  (1989)  and  is  reproduced 
in  Table  1-1. 

The  issue  of  anthropomorphism  in  the  interface  has  been  confounded 
by  the  dichotomy  between  personification  of  the  computer  (e.g.  "Please 
wait  while  I  sort  the  list")  and  the  use  of  actual  anthropomorphic  agents 
such  as  the  one  in  this  study.  Several  researchers  (Shniedermann,  1987; 
Turkle,  1984)  find  the  former  undesirable  and  even  deceptive.  The 
character-based  agent  provides  a  backdrop  for  the  powers  of  inflection  and 
human  dramatization  and,  moreover,  for  multiple  representations  of  the 


7 


same  communication.  Incorporating  these  human  elements  into  the 
interface  may  actually  make  the  computer  itself  less  conspicuous,  more 
invisible,  and  ultimately  more  of  a  medium. 

Table  1-1 

"Naturalness"  In  Human  Computer  Interaction 


Concept 


Reference 


System  should  provide  an  image  of  its 
underlying  processes  to  the  user. 

Normal  communication  between  individuals 
is  examined  to  apply  to  the  interface. 

Formal  communication  theory  applied  to 
human  computer  communication. 

Constructive  nature  of  human-human 
communication  applied  to  man-computer 
interaction. 

Human  and  computer  are  seen  as  partners. 

Interface  that  is  neither  too  under- 
determining  nor  too  over-determining. 

Computer  accepts,  analyzes,  and  critiques 
user  plans. 

System  made  more  natural  by  being  self- 
adaptive. 

Graceful  natural  language  interaction. 
Natural  artificial  languages. 


Fitter,  1979 
Kennedy,  1974 
Oberquelle  et  al.,  1983 
Thomas,  1978 

Press,  1971 
Thimbleby,  1980 

Langlotz  &  Shortcliff,  1983 

Innocent,  1982 

Hayes  &  Reddy,  1983 
Perlman,  1984 


Source:  Eberts  &  Eberts  (1989) 
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Brenda  Laurel  (1990)  claims  that  in  order  to  serve  the  user  best  a 
system  must  provide  (or  be  able  to  generate)  multiple  representations  as 
well  as  metaknowledge,  or  knowledge  about  problem  solving  in  a  domain. 
Susann  Brennan  observes, 

Multiple  representations  increase  the  odds  that  the  user  and  the 
system  will  be  able  to  communicate  effectively  and  that 
ambiguities  in  one  representation  will  be  disambiguated  by 
another.  (Laurel,  1990  p.  362) 

Formerly  ignored  aspects  of  user  interfaces  are  now  central  in  the 
thinking  of  experts  like  Patricia  Wright  of  the  Media  Research  Center 
Applied  Psychology  Unit,  Cambridge,  England,  who  claims  that 
"characteristics  of  the  interface,  in  the  broader  sense  of  both  the 
functionality  provided  and  the  form  of  its  display,  are  not  optional  extras, 
but  can  have  significant  consequences"  (Wright,  1989,  p.  282).  As  the 
technology  expands  to  include  audio  and  video  output,  as  well  as  guide  and 
other  on-line  help  platforms,  there  will  likely  be  an  increase  in  research 
focusing  on  these  design  characteristics. 

Interface  design  characteristics,  formerly  the  domain  of  educational 
software  designers,  are  increasingly  being  analyzed  by  educators  wishing  to 
incorporate  software  in  their  teaching.  In  Mindstorms,  Papert  (1980)  wrote 
about  "the  ways  that  choices  made  by  educators,  foundations, 
governments,  and  private  individuals  can  affect  the  potentially 
revolutionary  changes  in  how  children  learn"  (p.  32). 
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Today,  as  Papeit  predicted,  computer  technology  is  being  used  to 
augment  teaching  and  learning  at  many  levels.  Wenger  (cited  in  Ohlsson, 
1987,  p.  215)  reminds  us  that  one  function  of  teaching  is  the  act  of 
knowledge  communication,  even  when  it  is  carried  out  by  an  artifact. 
Binsted  (1987)  suspects  that  "learners  hold  strong  stereotypes  about  tutor 
roles,  so  that  when  a  talking  head  gives  input  or  process  information  they 
see  this  as  quite  congruent  with  the  stereotype"  (p.  39).  Given  the 
emergence  of  desktop  video  capability  in  personal  computers,  teachers 
themselves  may  eventually  become  guides  in  their  own  productions. 

The  idea  of  electronic  tutoring  in  general  or  of  "forming  an 
intellectual  partnership  between  the  student  and  tool"  (Salomon,  Perkins,  & 
Globerson,  1991)  has  been  the  subject  of  many  studies  in  the  realm  of 
artificial  intelligence  (for  a  survey  of  such  systems,  see  Woolf,  1988). 
Salomon  (1988)  found  that  several  functions  found  in  computer  tools  may 
help  build  such  an  intellectual  partnership,  including  (a)  assuming  some  of 
the  intellectual  burden  in  experimentation,  (b)  presenting  learners  with 
novel  alternatives,  (c)  freeing  the  individual  from  lower  level  operations, 
(d)  displaying  intermediate  processes,  (e)  providing  intelligent  tutelage,  and 
(f)  providing  models  of  information  representation,  processes  and 
strategies.  Conditions  necessary  for  internalization  of  such  functions  are 
also  described:  (a)  comprehensibility,  (b)  generalizability,  (c)  novelty,  (d) 
utility,  (e)  explicitness,  and  (f)  mindful  abstraction. 
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In  some  form,  all  of  Salomon's  functions  relate  to  on-line  help, 
especially  as  it  relates  to  activity  in  a  problem  space  (i.e.,  the  problem 
presented  to  the  user),  including  the  function  regarding  intelligent  tutelage. 
On-line  help  systems  often  employ  some  intelligent  decision-making 
features.  In  this  study,  for  example,  on-line  help  was  given  based  on 
position  in  the  problem  space,  previous  help  given,  and  an  analysis  of 
errors  being  committed.  Currently,  there  is  a  great  deal  of  research 
activity  in  this  area,  but  "current  research  on  intelligent  advisory  help 
facilities  has  not  adequately  addressed  their  utility  and  usability  as  a 
research  issue"  (Carrol  &  McKendree,  1987). 

The  study  of  Human-Computer  Interaction  as  a  discipline  is  attempting 
to  provide  system  designers  with  a  framework  for  evaluating  design.  By 
applying  science  through  application  "representations"  such  as  the 
interaction  scenario  used  in  this  study,  theories  may  be  developed  and 
tested  for  applicability. 

Background  of  the  Studv 
There  are  a  multitude  of  approaches  to  the  concept  of  electronic 
problem  solving.  Many  of  these  approaches  attack  the  point  between  the 
user's  view  of  the  system  and  the  functional  or  logical  view  of  the  system 
held  by  the  system  designer. 

Several  researchers  (Hammond  &  Barnard,  1984;  Ramey,  1989; 
Wright,  1989)  who  study  computer  user  interaction  point  out  the  difficulty 
between  using  "instrumental"  or  process-related  messaging  (messages  that 
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do  work,  as  in  command  language)  and  task-related  commentary. 
Ambiguities  in  the  presentation  itself,  called  Escher  effects,  which  force 
users  to  spend  substantial  time  sharing  attention  between  the  task  and 
rhetorical  analysis,  also  complicate  the  design  of  help  intervention.  This 
suggests  that  it  may  be  helpful  (to  the  user)  to  separate  the  procedural 
messages  and  the  more  global  metacognitive  or  strategy-related  messages  in 
on-line  help. 

The  type  of  intervention  supplied  the  user  in  a  problem  space  may 
have  a  significant  effect  on  the  entire  mediated  learning  experience.  Dr. 
Reuven  Feuerstein  (1990)  claims  that 

the  Mediated  Learning  Experience  is  responsible  for  a  more 
meaningful  and  generalized  type  of  change  that  actually  assumes  a 
structural  nature.  .  .  .  Learners  can  benefit  not  only  from  the 
direct  exposure  to  a  particular  stimulus,  but  they  can  also  forge  in 
themselves  a  repertoire  of  dispositions,  propensities,  orientations, 
attitudes,  and  techniques  that  enable  them  to  modify  themselves  in 
relation  to  other  stimuli,  (p.  75) 

According  to  Norman  (1991),  in  a  mediated  interaction  scenario  "actions 
are  performed  through  a  feedback  mechanism  involving  both  an  execution 
and  an  evaluation  phase.  Both  phases  of  the  action  cycle  need  support  from 
the  representational  format  used  by  the  artifact"  (p.  23).  In  the  action 
cycle,  the  interaction  centers  around  the  gulf  between  the  user's  goals,  or 
what  the  user  would  like  to  happen  and  the  environment  or  what  Norman 
calls  "The  World."  Focusing  on  discrepancies  in  the  knowledge  base  as 
critical  to  problem  solving  stems  from  Otto  Selz's  post-Gestalt  work  on 
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productive  thinking  and  learning  from  error.  Norman  (1991)  defines  the 
situation  as  follows: 

The  gulf  of  execution  refers  to  the  difficulty  of  acting  upon  the 
environment  (and  how  well  the  artifact  supports  those  actions). 
The  gulf  of  evaluation  refers  to  the  difficulty  of  assessing  the  state 
of  the  environment  (and  how  well  the  artifact  supports  the 
detection  and  interpretation  of  that  state),  (p.  23) 

Bodker's  (1989)  description  of  the  action  cycle  includes  the  idea  of 
"activity  flow."  When  a  consistent,  cohesive  activity  flow  occurs  between 
user  and  task,  there  is  a  feeling  on  the  part  of  the  user  of  direct 
engagement,  and  the  process  becomes  automated.  Bodker  (1989)  and 
Norman  (1991)  believe  that  interruptions  in  the  activity  flow  are  often 
disruptive  of  efficient  performance. 

It  is  not  clear,  however,  whether  help  intervention  is  identified  as  an 
interruption  or  an  intrinsic  part  of  the  human,  task,  artifact  system. 
Norman  (1991)  states. 

Artifacts  do  not  actually  change  an  individual's  capabilities. 
Rather,  they  change  the  nature  of  the  task  performed  by  the 
person.  When  the  informational  and  processing  structure  of  the 
artifact  is  combined  with  the  task  and  the  informational  and 
processing  structure  of  the  human,  the  result  is  to  expand  and 
enhance  cognitive  capabilities  of  the  total  system  of  human,  task, 
and  artifact,  (p.  19) 

The  human,  task,  and  artifact  system  or,  more  simply,  augmentation 
system  (Engelbart,  1963)  and  its  relationship  to  specific  help  intervention  is 
at  the  core  of  the  present  study.  In  order  to  analyze  the  activity  of  the 
augmentation  system  itself,  a  computer  program  has  been  developed  to 
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track  all  help-related  events.  Help  intervention  which  attempts  to  bridge 
the  gulf  of  execution  as  well  as  the  gulf  of  evaluation  is  supplied.  In  one 
treatment  the  help  offered  is  proactive,  that  is,  offered  without  request 
based  on  perceived  need.  In  the  second  treatment,  the  same  help  is 
available  on  user  request. 

Purpose  of  the  Study 

This  study  was  designed  to  determine  if  there  are  differences  in 
problem-solving  outcomes  between  experienced  and  nonexperienced 
learners  with  varying  locus-of-control  orientations  due  to  the  effect  of 
varied  types  of  digitized  video  guide  assistance  available  in  a  problem 
space.  In  addition,  the  study  examined  the  perceptions  of  the  subjects 
associated  with  the  types  of  digitized  video  guide-based  help  intervention, 
problem  difficulty  level,  and  usefulness  of  the  help  facilities. 

Subjects  and  Procedures 

Data  were  collected  on  105  subjects  enrolled  in  the  Introduction  to 
Instructional  Computing  course  of  the  College  of  Education,  University  of 
Florida.  The  subjects  were  tested  over  three  terms  of  1993.  Each  subject 
responded  to  the  Nowicki-Strickland  Locus  of  Control  Inventory,  a  one- 
page  computer  experience  survey  (Appendix  B),  and  computerized  version 
of  the  problem-solving  activity  "Coin  Challenge"  designed  by  Dr.  Elroy 
Bolduc  of  the  University  of  Rorida.  The  problem- solving  aspect  of  the 
treatment  also  contained  eight  attitudinal  questions  which  incorporated  a 
semantic  differential  scale  (Osgood,  Soci,  &  Tannenbaum,  1957).  The 
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experience  survey  and  the  locus  of  control  measures  were  pencil  and  paper 
implementations  and  undertaken  in  a  supervised  classroom  setting.  The 
computerized  portion  of  the  treatment  was  carried  out  in  a  computer  lab 
outfitted  with  17  Macintosh  LCII  computers.  Subjects  were  randomly 
placed  at  computers  alternately  set  up  to  offer  proactive  system-initiated 
help  or  manual  on-demand  help  for  the  first  activity.  During  the  second 
activity  the  type  of  help  offered  was  switched.  After  the  completion  of  the 
second  activity,  the  subjects  were  given  a  preference  indicator  and  asked  if 
they  would  like  to  do  the  activity  a  third  time.  Those  that  did  were  allowed 
to  choose  the  type  of  help  intervention  used.  An  audit  trace  of  all  subjects' 
activity  was  kept  by  the  Coin  Challenge  program. 

Research  Questions 
The  research  questions  addressed  in  this  study  fall  into  three  categories 
designed  to  investigate  the  somewhat  broader  questions: 

1.  Does  digital  video  guide-based  help  intervention  support  a  user's 
efforts  in  a  problem  space? 

2.  Is  the  effectiveness  of  proactive  digital  video  guide-based  help 
affected  by  parameters  such  as  locus  of  control  and  Experience? 

3.  What  factors  contribute  to  the  user's  perception  of  the  video  guide- 
based  help  system? 

Data  were  gathered  from  a  trace  of  the  subjects'  behavior  during  the 
problem-solving  activity  and  from  responses  to  attitudinal  questions.  The 
attitudinal  questions  queried  the  participants  as  to  whether  or  not  they 
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enjoyed  the  problem- solving  activity,  whether  or  not  they  found  the  video 
guide-based  help  useful  and  which  type  of  guide-based  help  they  preferred. 
Variables  in  the  study  were  analyzed  using  the  predictors:  locus  of  control 
index,  computer  experience  index,  proactive  help  status,  perceptions  of 
their  own  problem- solving  ability,  estimations  of  the  difficulty  level  of  the 
problem,  and  the  associated  interactions. 

The  following  research  questions  were  addressed  by  this  study: 

1.  Is  there  a  significant  difference  in  achievement  (i.e.,  the  ability  to 
solve  mathematical-logic  problems)  when  the  amount  of  video  guide-based 
help  intervention  is  varied? 

2.  What  relationship  does  the  amount  of  help  requested  have  to 
achievement? 

3.  Does  proactive,  digital  video  guide-based  help  increase  the 
likelihood  that  undergraduate  students  will  request  help  more  often? 

4.  Is  there  a  significant  difference  between  undergraduate  students 
with  varying  locus-of-control  indices  in  their  ability  to  solve 
mathematical-logic  problems  when  the  type  of  video  guide-based  help 
intervention  is  varied? 

5.  Is  there  a  significant  difference  in  achievement  due  to  an  interaction 
effect  of  locus-of-control  orientation  and  the  type  of  video  guide-based 
help  intervention? 
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6.  Is  there  a  significant  difference  in  achievement  between  students 
with  varying  computer  experience  when  the  type  of  video  guide-based  help 
intervention  is  varied? 

7.  Is  there  a  significant  difference  in  achievement  due  to  an  interaction 
effect  between  type  of  video  guide-based  help  intervention  and  prior 
computer  experience? 

8.  Is  there  a  significant  difference  in  achievement  when  the  type  of 
video  guide-based  help  intervention  is  varied  due  to  an  interaction  effect  of 
locus  of  control  orientation  and  prior  computer  experience? 

9.  Is  there  a  significant  difference  in  undergraduate  students' 
preference  towards  proactive  help  due  to  their  estimation  of  their  own 
problem- solving  ability? 

10.  Is  there  a  significant  difference  in  undergraduate  students' 
preference  towards  proactive  video  guide  help  based  on  their  estimation  of 
the  difficulty  of  problems  presented  to  them? 

11.  Is  there  a  significant  difference  in  preference  towards  proactive 
video  guide  help  due  to  the  interaction  between  an  estimation  of  their  own 
problem-solving  ability  and  their  estimation  of  the  difficulty  of  problems 
presented  to  them? 

12.  Are  there  significant  differences  in  preference  toward  type  of 
video  guide-based  help  intervention  due  to  the  effects  of  their  locus-of- 
control  orientation,  prior  computer  experience,  and/or  the  interaction 
between  locus  of  control  and  experience? 
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13.  Are  there  significant  differences  in  the  expressed  enjoyment  of  the 
problem-solving  activity  based  on  the  effects  of  locus  of  control 
orientation,  prior  computer  experience,  perception  of  problem-solving 
ability,  perception  of  the  problem's  difficulty,  and/or  the  delivery  system's 
proactive  help  status? 

14.  Are  there  significant  differences  in  perceived  usefulness  of  video 
guide  help  based  on  the  effects  of  locus  of  control  orientation,  experience, 
perception  of  problem-solving  ability,  perception  of  the  problem's 
difficulty,  and/or  the  delivery  system's  proactive  help  status? 

Limitations  of  the  Study 

This  study  contained  limitations  that  affect  the  generalizability  of  the 
results.  This  study  was  conducted  with  undergraduate  students  enrolled  in 
an  instructional  computing  course  designed  for  education  majors. 
Enrollment  in  the  course  is  not  limited  to  education  majors  but  is  a 
required  course  for  elementary  education  majors.  Only  those  students  who 
consented  to  participate  were  included. 

The  problem-solving  activity  at  the  core  of  this  study,  although 
designed  and  developed  locally,  has  several  characteristics  in  common  with 
Mastermind™,  a  commercial  game  with  widespread  availability.  Subjects 
familiar  with  this  game,  or  games  similar  to  it,  would  have  an  advantage 
over  subjects  who  had  never  encountered  such  a  game  and,  therefore, 
might  need  to  rely  less  on  the  help  features  under  consideration.  Also, 
since  the  subjects  were  studying  instructional  computing,  a  professional 
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interest  in  the  activity  might  have  been  invoked  by  the  treatment.  The 
results  of  this  study  are  generalizable  to  situations  involving  similar  stimuli 
and  classes  of  variables. 

Definition  of  Terms 
Agent  as  the  term  applies  to  computing  can  be  defined  as  a  character, 
enacted  by  the  computer,  who  acts  on  behalf  of  the  user  in  a  virtual 
(computer-based)  environment"  (Laurel,  1990,  p.  356).  The  idea  of  an 
agent  was  first  conceived  by  John  McCarthy  of  M.I.T.  in  the  mid-1950s. 
The  term  "agent"  was  coined  a  few  years  later  by  Oliver  G.  Selfridge 
(Kay,  1984). 

Guide  will  be  used  in  this  study  to  mean  a  domain-specific  agent  whose 
abilities  will  be  confined  to  providing  context-sensitive  help  to  a  user.  The 
"guide"  metaphor  originally  emerged  as  a  travel  "agent"  in  the  process  of 
designing  an  interface  for  an  educational  Hypermedia  database  (Oren, 
Salomon,  &  Kreitman,  1990).  The  guide  used  in  this  study  is  animated 
(digitized  video)  and  speaks  to  the  user.  Although  they  are  available  upon 
request,  they  may  also  provide  unsolicited  information.  That  is,  they  may 
be  "proactive." 

Heuristic  will  be  used  to  mean  a  general  suggestion  or  strategy, 
independent  of  any  particular  topic  or  subject  matter,  that  helps  problem 
solvers  approach  and  understand  a  problem  and  efficiently  marshal  their 
resources  to  solve  it  (Schoenfeld,  1980). 
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Human-Computer  Interaction  (HCI)  is  an  interdisciplinary  area  of 
applied  research  and  design  practice.  Its  key  concern  is  to  understand  and 
facilitate  the  creation  of  "user  interfaces,"  that  is,  of  computers  as 
experienced  and  manipulated  by  human  users  (Carroll,  1991,  p.  1). 

On-line  help  is  a  user  support  structure  provided  by  a  computer 
providing  information  related  to  the  task  at  hand  or  the  program  being 
used.  It  may  or  may  not  be  context  sensitive  and  its  design  is  usually 
handled  by  the  creators  of  the  application  program. 

Locus  of  Control  (LOC)  is  a  construct  which  comes  from  social 
learning  theory.  It  refers  to  a  person's  perceptions  of  the  agency  of 
control  of  the  reinforcement  he/she  receives.  One  is  said  to  have  an 
internal  locus  of  control  if  one  feels  that  the  reinforcements  one  receives 
occur  primarily  because  of  his/her  own  purposeful  behavior.  One  is  said 
to  have  an  external  locus  of  control  if  one  feels  that  the  reinforcements  one 
receives  occur  primarily  because  of  forces  beyond  the  individual's  control 
(Butterfield,  1964.) 

Mediated  Learning  Experiences  (MLE)  are  interactions  during  which 
the  human  organism  is  subject  to  the  intervention  of  a  mediator.  Learners 
can  benefit  not  only  from  the  direct  exposure  to  a  particular  stimulus,  but 
they  can  also  forge  in  themselves  a  repertoire  of  dispositions,  propensities, 
orientations  (e.g.,  locus  of  control),  attitudes,  and  techniques  that  enable 
them  to  modify  themselves  in  relation  to  other  stimuli  (Fuerstein,  1990, 
p.  75). 
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Video  guide  is  used  to  mean  a  digital  video  representation  of  a  guide  as 
mentioned  above.  The  digital  video  was  created  by  "digitizing"  an  analog 
video  recorded  in  a  video  studio,  using  a  live  performer  acting  as  the 
guide. 

Summary 

The  field  of  instructional  technology  is  evolving  rapidly  and  now 
represents  a  convergence  of  theories  from  the  fields  of  information 
science,  contemporary  psychology,  human-computer  interaction,  computer 
science,  media,  and  the  traditional  disciphnes  associated  with  the  field  of 
education.  This  chapter  presents  a  description  of  the  problem,  a  need  for 
the  study,  an  overview  of  related  literature,  a  list  of  research  questions 
addressed  by  the  study,  selected  definitions,  and  Umitations  and 
dehmitations  of  the  study. 

Chapter  2  contains  a  review  of  literature  that  bears  on  questions 
investigated  in  the  study.  It  focuses  on  two  domains  of  machine-mediated 
learning  as  they  relate  to  on-line  help  and  guide-based  help  in  particular, 
and  on  social  learning  theory,  for  example,  locus  of  control  orientation  as  a 
mediating  variable.  A  description  of  the  intervention  tool  and  the  design  of 
the  study  appears  in  Chapter  3.  Chapter  4  presents  the  findings  of  the 
study.  Chapter  5  discusses  the  findings,  their  implications  for  instructional 
design,  and  recommendations  for  future  research. 


CHAPTER  2 
REVIEW  OF  THE  RELATED  LITERATURE 

Cognitive  Artifacts 

The  interface  between  the  computer  and  a  human  user  illustrates  a 
dovetail  of  direct  information  and  representational  information.  Don 
Norman  (1988)  has  suggested  that  human-computer  interfaces  enable  and 
represent.  In  other  words,  the  computer  and  the  user  both  have  a  role  in 
the  task  at  hand.  As  Norman  (1991)  has  noted,  putting  computers  or  other 
"cognitive  artifacts"  between  a  human  and  a  task  does  not  actually  change 
the  individual's  capabilities,  instead  it  changes  the  nature  of  the  task 
performed  by  the  person.  Duffy,  Mehlenbacher,  and  Palmer  (1989,  p. 
367)  argue  that  in  such  task-artifact  scenarios  it  is  necessary  to  understand 
(a)  how  the  user  accomplishes  tasks  with  the  artifact,  (b)  how  the  user  deals 
with  breakdowns  in  task  performance,  and  (c)  how  the  help  system  can  get 
the  user  back  on  track.  Users  will  not  try  to  access  performance-related 
information  as  long  as  the  task  is  proceeding  smoothly.  Winograd  and 
Flores  (1986)  have  noted,  however,  that  all  users  have  breakdowns  in 
which  they  lack  either  the  machine  source  procedural  information  or 
domain  specific  semantic  information  to  further  the  task. 
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Theories  Related  to  the  Task- Artifact  Cvcle 
Beyond  descriptions  of  how  well  performance-oriented  systems 
support  user  actions  and  user  evaluation,  many  theories  exist  to  explain 
how  users  accommodate  and  interpret  system  feedback. 
Attention 

When  a  goal-directed  activity  breaks  down,  users  are  thrust  into  a 
secondary  level  of  problem  solving.  Users  must  diagnose  the  problem, 
determine  its  implications,  and  attempt  to  solve  it.  The  solution  may 
depend  on  many  factors  including  requisite  knowledge,  design  of  the 
artifact,  and  available  information  sources  (Duffy  et  al.,  1989;  Norman, 
1991).  Many  researchers  (Bodker,  1989;  Norman,  1991;  VanLehn,  1990) 
have  addressed  this  impasse  or  breakdown  in  the  activity  cycle.  At  such  a 
breakdown,  the  user's  attention  switches  to  the  impasse  or,  in  Norman's 
terms,  the  gulf  of  evaluation.  Although  attention  was  formerly  viewed  as 
having  limited  quantitative  capacity  (Moray,  1967,  referenced  in  Eberts  & 
Eberts,1989;  Kahneman,  1973),  some  later  theories  (e.g.,  McLeod,  1977; 
Navon  &  Gopher,  1979)  accommodate  the  processing  of  information  from 
several  sources,  often  referred  to  as  "resource  pools,"  thereby  allowing 
multiple  responses.  Wickens  (1980)  developed  a  model  for  processing 
resources  that  contains  three  dimensions.  Each  of  the  dimensions  draws 
upon  specific  resource  pools:  stage-defined  resource  pools  (eariy  and  late 
processes),  modality-defined  resource  pools  (auditory  and  visual  encoding), 
and  processing-defined  resource  pools  (spatial  and  verbal  processes).  The 
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stage-defined  dimension  allows  the  encoding  or  processing  of  information 
simultaneously  with  responses  to  previous  information.  The 
modality-defmed  dimension  contains  resource  pools  for  simultaneous  visual 
and  auditory  signal  processing.  The  processing-defined  pools  represent 
spatial,  graphic  information  and  verbal,  textual  forms  (Eberts  &  Eberts, 
1989).  According  to  Wickens  (1980)  and  Wickens,  Sandry,  and  Vidulich 
(1983),  users  are  less  likely  to  be  overloaded  if  information  is  presented  or 
time-shared  from  both  pools  in  a  dimension.  Other  researchers  (McLeod, 
1977;  Robinson,  Base,  &  Eberts,  1985)  have  validated  predictions  based  on 
Wickens'  synthesized  structure  for  processing  resources. 
Building  Mental  Models  and  the  Design  of  Production  Svstems 

Problem-solving  skills  and  inquiry  strategies  were  the  focus  of  several 
studies  in  the  early  1970s.  Robinson,  Tickle,  and  Brison  (1972)  developed 
a  basic  problem- solving  model  based  on  a  linear  progression  through  the 
following  steps:  (a)  identifying  a  question,  (b)  identifying  possible 
alternative  answers  or  avenues  of  inquiry  that  will  produce  an  answer,  (c) 
finding  information  about  the  alternatives,  (d)  adding  up  or  synthesizing 
this  information  in  some  way,  and  (d)  choosing  an  alternative. 

Human  Problem  Solving  (1972)  by  Newell  and  Simon  dealt  with  many 
aspects  of  problem  solving  and  stimulated  research  in  computer-related 
activities.  Theories  presented  in  the  book  were  based  on  the  authors'  earlier 
work  involving  problem-solving  techniques  or  "information  processes" 
(Newell,  Simon,  &  Shaw,  1958),  computer  programs  called  Logic  Theorist 
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(Newell  et  al.,  1958),  and  The  General  Problem  Solver  (Ernst  &  Newell, 
1969)  and  correlations  between  human  and  computer  concept  acquisition 
(Simon  &  Kotovsky,  1963).  Many  early  computer-related  studies  dealt 
with  error  correction  and  semantic  manipulations  (Coombs  &  Alty,  1984). 
These  studies  examined  the  use  of  sentence  tokens,  sign  and  symbol  use  in 
specific  behavioral  contexts,  and  interpretation  of  expressions  (Cresswell, 
1973;  Morris,  1971;  Montague,  1972,  respectively,  cited  in  Jackson  & 
Lefrere,  1984). 

As  these  studies  proliferated,  researchers  attempted  to  develop  a  theory 
of  cognition  which  accommodated  semantics,  interpretation,  and 
appropriateness  in  the  man-machine  interface  (Bartsch,  1979;  Kasher, 
1977;  Moran,  1981).  This  type  of  analysis  was  expanded  by  theorists 
Jackson  and  Lefrere  (1984)  to  accommodate  plan-based  engineering  of 
human-computer  systems.  According  to  Jackson  and  Lefrere  (1984),  there 
are  four  aspects  of  planning  relevant  to  man-machine  interaction: 

(1)  provide  planning  aids  for  the  user  at  the  formation  stage,  (2) 
recognize  the  user's  plan  as  an  instantiation  of  one  of  its  plan 
schemas,  (3)  revise  its  own  hypothesis  about  the  user's  plan 
(should  it  turn  out  to  be  false)  and  help  the  user  revise  his  plan 
(should  it  fail)  and  (4)  promote  the  generalization  of  plans  which 
succeed.  These  desiderata  are  easy  to  state,  but  difficult  to 
engineer,  (p.  71) 

Systems  based  on  plan-based  theories  require  self-regulating  facilities 
which  must  necessarily  make  decisions  about  user  intentions,  that  is,  the 
system  must  manipulate  the  representation  of  the  problem  space  in  ways 
that  support  the  user.  This  manipulation  has  been  referred  to  as  a 
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base-level  action  (Genesereth,  1983).  According  to  Genesereth,  choosing 
and  sequencing  base-level  actions  are  meta-level  actions  that  are  performed 
by  a  system  component  called  the  meta-level  interpreter.  In  other  words,  a 
highly  organized  system  is  supporting  a  user  with  a  highly  organized  plan 
involved  in  a  multi-leveled  problem- solving  scenario. 

Highly  organized  system  models  play  well  in  instructional  design 
circles,  and  several  instructional  designers  (Jones  &  Wipond,  1988; 
Merrill,  1987;  Pirolli  &  Russell,  1990)  have  developed  intelligent  tutoring 
systems  based  on  complex  instructional  models.  Conceptual  Systems 
Theory  developed  by  David  Hunt  and  associates  is  an  example  of  an  area  of 
research  based  primarily  on  the  structure  of  a  learner's  cognitive 
complexity  rather  than  the  content  of  personality  development  (Joyce, 
1980,  p.  19).  In  order  to  delineate  the  structure  of  these  complex  models, 
many  researchers  have  developed  sophisticated  schematic  representations. 
Anderson's  (1988)  review  of  inteUigent  tutoring  systems  discusses  methods 
of  representing  declarative  knowledge,  procedural  knowledge,  and 
qualitative  process  models  symbolically  in  order  to  promote  desirable 
learning  outcomes  better. 

Newell  and  Simon  (1972)  proposed  the  idea  of  a  production  system  as 
a  model  of  the  human  information  system.  Production  systems  have 
provided  an  almost  generic  theoretical  formalism  in  cognitive  psychology 
(Poison,  1987).  Card,  Moran,  and  Newell  (1983)  formalized  the  GOMS 
model:  goals,  operations,  methods,  and  selection  rules  in  order  to  analyze 
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a  learner's  "how-to  do-it"  knowledge.  In  the  GOMS  model,  goals  are 
organized  into  collections  of  interrelated  goals  which  are  manipulated  by 
control  structures  such  as  do-until  loops.  Goals  represent  user  intentions; 
operations  are  triggered  by  users'  basic  physical  actions;  methods  are 
composed  of  sequential  operations  necessary  to  achieve  a  subgoal;  and 
selection  rules  are  problem-solving  fragments  which  specify  when  to  apply 
a  given  method.  Card,  Moran,  and  Newell  (1983)  applied  the  GOMS 
model  to  the  task  of  manuscript  editing.  The  task  was  decomposed  into  a 
series  of  unit  tasks,  subgoals,  methods,  and  submethods.  It  is  important  to 
emphasize  that  a  user's  how-to-do-it  knowledge  is  a  mixture  of 
task-specific  and  interface-specific  information  (Poison,  1987). 

Kieras  and  Poison  (1985)  developed  a  more  elaborate  production 
system  model  involving  rules,  working  memory,  and  an  interpreter.  Rules 
are  specified  by  a  condition-action  pair  of  the  form  IF  (condition)  THEN 
(action).  The  working  memory  contains  representation  of  current  goals, 
notes  about  current  activities,  and  the  external  environment.  The 
interpreter  operates  in  cycles  alternating  between  recognize  and  act  modes 
(Poison,  1987). 

Although  instructional  designers  and  other  researchers  continue  to 
develop  complex  symbolic  representations  of  learner  development,  transfer 
and  productivity,  questions  concerning  the  validity  of  complex 
interpretations  of  production-system  activity  are  common  in  the  literature. 
A  production-system  model  generates  a  hypothetical  series  of  cognitive 
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operations  and  the  sequence  of  user  actions  required  to  complete  the  task. 
Upon  making  the  assumptions  necessary  for  representation  of  the  user's 
knowledge,  researchers  derive  learning  and  performance  predictions 
(Poison,  1987).  Pea  (1988),  however,  argues  that 

the  productions  in  production  systems  modeling  human  thought  or 
the  nodes  in  knowledge  representation  networks  in  cognitive 
simulations  are  part  of  the  theorist's  construction  of  the  problem 
simation.  The  question  is  begged  because  the  problem  solving  is 
interpreted,  not  an  experimental  variable  defined  invariantly 
across  subjects,  (p.  181) 

In  their  seminal  work  Human  Problem  Solving.  Newell  and  Simon  (1972) 
warned  of  putting  too  much  emphasis  on  interpretation: 

The  study  of  learning  takes  its  cue  from  the  nature  of  the 
performance  system.  If  performance  is  not  well  understood,  it  is 
somewhat  premature  to  study  learning.  ...  It  is  our  judgement 
that  in  the  present  state  of  the  art,  the  study  of  performance  must 
be  given  precedence,  even  if  the  strategy  is  not  costless,  (p.  7) 

The  difficulty  in  developing  mental  models  for  users  of  technology  is 
emphasized  by  Norman's  (1983)  findings.  In  observations  of  calculator 
users,  Norman  found  rather  than  following  a  sophisticated  plan,  user's 
mental  models  were  incomplete,  unstable,  unscientific,  and  parsimonious. 
For  example,  Norman  found  that  participants  often  wrote  down 
intermediary  steps  instead  of  using  the  calculator's  memory  and  that  they 
pressed  the  clear  key  several  times  when  a  single  keypress  achieved  the 
desired  outcome.  Elsewhere,  Norman  (1984)  summarizes,  "What  matters 
is  how  real  users  actually  perform,  not  what  designers  believe  the  ideal 
user  is  capable  of  doing"  (p.  14). 
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Carrol  and  McKendree  (1987)  agree  that  "there  is  little  research 
evidence  to  support  mental  models  of  complex  task  domains  and  virtually 
no  direct  behavioral  research  either  informs  claims  about  psychological 
pertinence  or  has  been  brought  to  bear  on  evaluating  these  claims"  (p.  17). 

Whiteside  and  Wilson  (1987)  insist  that  designers  should  assume  that 

users  do  not  start  with  goals: 

The  user's  goals  are  assumed  to  precede  behavior,  to  have  clear 
conditions  of  satisfaction,  to  be  interpretable  outside  of  the  larger 
context  of  the  user's  situation,  to  be  organized  in  a  strict 
hierarchy,  and  not  to  undergo  radical,  moment-to-moment 
transformation.  .  .  .  [This]  does  not  capture  what  we  observe  in  the 
field  or  lab,  in  our  studies  and  observations  of  users.  .  .  .  Users 
are  often  unable  to  state  unambiguous  conditions  of  satisfaction, 
and  are  dependent  on  the  context  in  which  they  are  operating.  To 
the  extent  that  we  can  identify  the  users'  goals  at  any  time,  they 
are  not  organized  in  any  strict  hierarchy  and  are  radically 
transformed  as  events  unfold,  (p.  359) 

Help  Intervention 
Modem  computerized  help  facilities  are  based  in  part  on  the 
formalization  of  the  terse,  cryptic  error  messages  common  in  the  early 
days  of  the  personal  computer  (Brown,  1983).  Error  messages,  unlike 
help  messages,  are  generated  only  when  an  error  state  has  been  reached 
(Bosser,  1987).  Shneiderman's  (1982)  efforts  brought  a  set  of  informal 
guidelines  for  error  messages  to  the  areas  of  programming  and  word 
processing.  In  addition,  early  work  in  advice-giving  systems  (Alty  & 
Coombs,  1981)  and  varying  approaches  to  help  intervention  (Fischer, 
Lemke,  &  Schwab,  1985)  studied  user  behavior  and  the  effects  of  concise 
help  messaging. 
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John  Waterworth  (1984)  developed  a  model  for  interaction  as 
conversation  based  on  the  work  of  Bunt,  Leopold,  Muller,  and  van  Katwijk 
(cited  in  Waterworth,  1984)  who  drew  a  distinction  "between  two  types  of 
dialogue  acts:  goal-oriented  acts  and  dialogue-control  acts.  The  former 
involve  the  direct  communication  of  goal-relevant  information  between 
system  and  user;  the  latter  are  implicated  in  the  prevention  and  correction 
of  mistakes"  (p.  231).  Waterworth's  model  expands  this  distinction  into 
three  types  of  dialogue  acts:  goal-oriented  acts,  dialogue-control  acts,  and 
other  acts.  Goal-oriented  acts  compose  the  main  body  of  the  task-related 
dialogue,  while  dialogue-control  acts  include:  confirmation, 
recovery/repair,  help  safety  Net,  extra  guidance,  stop  information, 
restart/repeat,  and  exit.  Other  dialogue  acts  include  the  prologue  and 
epilogue  to  the  task  at  hand.  The  present  study  applied  this  model  with 
respect  to  interaction. 

Vygotsky  (1978)  uses  the  term  Zone  of  Proximal  Development  to 
describe  steps  or  levels  of  understanding  which  can  be  encouraged  to 
maturity  by  the  timely  presentation  of  informative  material  which 
confirms,  clarifies,  or  relates  to  what  the  individual  already  understands. 
The  present  study  attempts  to  take  advantage  of  the  user's  immediate 
experience  by  providing  concise,  informative  feedback  which  is  sensitive  to 
situational  context.  According  to  Waterworth  (1984),  "Because  of  the 
temporary  nature  of  speech,  and  the  limitations  of  human  memory, 
information  must  be  presented  in  small  enough  'packets'  to  be  easily 
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assimilated  and  acted  upon  by  the  user"  (p.  234).  Long,  complete  dialogue 
has  been  avoided  in  favor  of  small,  suggestive  fragments.  The  design, 
thereby,  affords  the  user  what  Andrea  diSessa  (1986)  has  termed  a 
"continuous  incremental  advantage"  (p.l  38).   According  to  diSessa,  "The 
importance  of  the  principle  is  not  only  having  small  steps  in  learning,  and 
motivation  for  those  steps,  but  so  that  at  each  stage  the  learner  gets 
structured  feedback  on  competent  performance,  feedback  in  terms  of 
understanding  goals"  (p.  139).  Similarly,  researchers  Knox,  Bailey,  and 
Lynch  (1989)  discovered  the  need  for  what  they  call  the  "sequential 
disclosure"  technique  in  interface  design.  "This  technique  was  developed  as 
we  found  that  complete  answers  to  subject  queries  often  obscured  the 
identification  of  key  design  features  which  could  enable  insight  on  the  part 
of  the  subject"  (p.  284). 
Norman  (1984)  has  said. 

Just  as  we  have  no  need  for  an  "error  message"  in  conversation 
there  should  be  no  such  thing  as  an  "error  message"  in  computer 
interaction.  Rather,  each  utterance  should  be  thought  of  as 
another  iteration  toward  the  goal,  and  if  it  is  incompletely  formed, 
ambiguous,  or  uninterpretable,  nonetheless  it  represents  an  honest 
attempt  toward  the  goal.  Feedback,  comments,  and  offers  of 
assistance,  yes;  error  messages,  no.  Recognition  of  this  point  of 
view  changes  the  nature  of  the  dialogue  considerably,  (p.  13) 

Schoen  (1982)  notes. 

When  the  practitioner  tries  to  solve  the  problem  he  has  set,  he 
seeks  both  to  understand  the  situation  and  to  change  it.  .  .  . 
Through  the  unintended  effects  of  action,  the  situation  talks  back. 
The  practitioner,  reflecting  on  this  back-talk,  may  find  new 
meaning  in  the  situation  which  leads  him  to  new  reframing.  Thus 
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he  Judges  a  problem-setting  by  the  quality  and  direction  of  the 
reflective  conversation  to  which  it  leads,  (pp.  134-135) 

An  important  feature  of  modem  computer-based  interfaces  is  the 
ability  of  one  part  of  the  user-machine  dialogue  to  refer  to  another. 
Draper  (1986)  refers  to  this  ability  as  inter-referential  I/O  and  argues  that 
it  is  the  aspect  of  human  conversation  which  allows  us  to  cross  the  gap 
between  monologue  and  dialogue.  In  man-machine  dialogues,  the  machine 
does  not  originate  communication  but  simply  translates  actions  by  user  and 
system  into  changes  to  the  display  and  maintains  the  necessary  records  for 
interpreting  later  references  (p.  347).  The  interface  is  then  able  to  enter 
the  domain  of  discourse  between  the  user  and  the  system.  The  present  study 
involves  the  application  of  inter-referential  techniques  with  respect  to  help 
intervention. 

Draper  (1986)  argues  that  a  change  in  appearance  in  the  interface  may 
not  only  "alter  the  total  size  and  amount  of  information  displayed,  but  also 
its  nature-that  is,  to  present  an  alternative  perspective  on  the  same  object" 
(p.  343).  Gombrich  (1974)  agrees. 

The  easier  it  is  to  separate  the  code  from  the  content,  the  more  we 
can  rely  on  the  image  to  communicate  a  particular  kind  of 
information.  A  selective  code  that  is  uncferstood  to  be  a  code 
enables  the  maker  of  the  image  to  filter  out  certain  kinds  of 
information  and  to  encode  only  those  features  that  are  of  interest 
to  the  recipient.  Hence  a  selective  representation  that  indicates  its 
own  principles  of  selection  will  be  more  informative  than  the 
replica,  (p.  254) 

The  system  under  study  uses  a  digitized  video  guide  in  an  otherwise  static 
graphic  environment. 
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Proactive  Help:  User-Initiated  vs.  Svstem-Initiated  Help 
Modem  computer  systems  provide  the  tools  required  to  measure  user 
activity  both  temporally  and  functionally.  In  addition  to  allowing  for 
sophisticated  reporting  of  user  activity,  this  feature  allows  the  help 
facilities  to  become  pro-active.  Draper  (1986)  suggests,  "Another 
important  technique  is  that  of  autodisplay-of  displaying  output 
automatically  without  specific  user  request.  .  .  .  Communication  takes  place 
by  user  and  system  alternately  modifying  and  referring  to  the  objects  on 
the  screen"  (p. 347).  System-initiated  (autodisplayed)  help  is  central  to  the 
current  study. 

Cohil  and  Williges  (1982)  compared  user-initiated  help  to  system- 
initiated  help  in  a  text-editing  task  scenario.  It  was  found  that  the  best 
performance  was  achieved  with  user-initiated  help  which  referred  to 
off-line  documentation. 

Miyata  and  Norman  (1986)  have  developed  a  set  of  guidelines  to  be 
applied  to  interrupt-driven  activities  and  system  messaging:  "It  is  clear  that 
an  interrupting  message  should  not  be  presented  in  the  stages  of  execution 
or  perception,  stages  where  the  user  is  directly  interacting  with  the 
system.  ...  A  good  procedure  might  be  to  present  relatively  unobtrusive 
reminders  to  the  user  just  following  the  completion  of  an  action"  (p.  278). 
The  model  used  in  the  present  study  relates  procedural  messages 
immediately  following  an  inappropriate  action. 
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Locus  of  Control 

Since  the  first  locus-of-control  scale  was  developed  by  Rotter  in  1966, 
locus-of-control  scales  have  been  developed  by  several  researchers 
(Goodman  &  Waters,  1987).  The  original  scale  developed  by  Rotter  was 
enhanced  by  Reid  and  Ware  in  1974  to  measure  three  dimensions  of  locus 
of  control:  fatalism,  social  systems,  and  self  control.  Alternatively,  in 
1974  Levenson  developed  the  three  locus-of-control  dimensions:  chance, 
powerful  others,  and  internal.  Nowicki  and  Strickland  developed  a  locus- 
of-control  scale  for  children  in  1973  using  a  fifth-grade  reading  level  and 
later  modified  the  scale  for  adults  (Lefcourt,  1976). 

Rotter  (quoted  in  Louie,  Luick,  &  Louie,  1986)  explains, 

When  a  reinforcement  is  perceived  by  the  subject  as  following 
some  action  of  his  own  but  not  being  entirely  contingent  upon  his 
action,  then,  in  our  culture,  it  is  typically  perceived  as  the  result  of 
luck,  chance,  fate,  as  under  the  control  of  powerful  others,  or  as 
unpredictable  because  of  the  great  complexity  of  the  forces 
surrounding  him.  When  the  event  is  interpreted  in  this  way  by  an 
individual,  we  have  labeled  this  a  belief  in  external  control.  If  the 
person  perceives  that  the  event  is  contingent  upon  his  own 
behavior  or  his  own  relatively  permanent  characteristics,  we  have 
termed  this  a  belief  in  internal  control,  (p.  102) 

The  first  study  linking  locus  of  control  and  cognitive  activity  was 
conducted  by  Seeman  and  Evans  in  1962  (Lefcourt,  1976).  These 
researchers  determined  that  internals  seek  out  information  even  if  it  has 
negative  connotations  for  themselves.  Seemam  (1963,  cited  in  Lefcourt, 
1976)  tested  these  assertions  and  concluded  that  an  individual's  sense  of 
powerlessness  governs  his  attention  and  acquisition  processes. 
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Several  educational  researchers  have  studied  relationships  between 
locus  of  control  and  the  style  of  instruction  (Shadbolt,  1978).  Leith  (1973) 
argued  that  there  was  a  relationship  between  teaching  strategy  and  the 
extrovert-introvert  dimension  of  personality,  where  extroverts  achieved 
higher  scores  on  a  criterion  measure  after  following  an  unstructured  style 
of  teaching,  while  introverts  achieved  higher  success  with  a  clearly 
structured  style  of  teaching.  According  to  Shadbolt  (1978),  "Perhaps  the 
most  difficult  variable  to  evaluate  is  teaching  strategy"  (p.  227).  The 
present  study  attempts  to  quantify  the  teaching  strategy  variable  by 
computerized  control  of  the  guide-based  intervention. 

Coleman,  Campbell,  Hobson,  McPartland,  Mood,  Weinfeld,  and  Work 
(cited  in  Louie,  Luick,  &  Louie,  1986,  p.  104)  studied  almost  half  a  million 
school  children  across  the  United  States  and  found  that  a  belief  of  destiny 
was  a  major  factor  in  school  achievement. 

According  to  Prawat,  Grissom,  and  Parish  (1979),  persons  who 
attribute  their  success  or  failure  to  fate,  chance,  or  other  people  are 
designated  as  "externals";  "internals,"  on  the  other  hand  are  people  who 
accept  personal  responsibility  for  what  happens  to  them.  In  a  smdy  of  499 
students  in  grades  3-12,  the  researchers  found  that  females  were  more 
internally  controlled  than  males,  with  a  dramatic  decrease  in  externality 
noted  in  the  middle  school  group  (both  males  and  females).  A  further 
decrease  in  externality  was  found  in  the  high  school  group.  Wildstein 
(1982)  applied  the  same  locus-of-control  measure  (the  Children's 
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Nowicki-Strickland  instrument)  to  144  children  and  found  that  internals 
scored  significantly  higher  on  verbal  problem-solving  measures  than 
externals  but  found  no  sex  differences. 

Butterfield  (1964)  found  that  frustration  reactions  become  less 
constructive  as  locus  of  control  becomes  more  external  and  that 
intropunitive  reactions  become  more  frequent  as  locus  of  control  becomes 
more  external.  Interestingly,  Butterfield  found  no  relationship  between 
locus  of  control  and  extrapunitive  reactions.  Butterfield  also  studied 
anxiety  reactions  with  respect  to  locus  of  control  and  found  that  debilitating 
anxiety  reactions  increased  and  facilitating  anxiety  reactions  decreased  as 
locus  of  control  became  more  external.  In  other  words,  the  more  the 
subject  felt  control  over  what  happened,  the  more  likely  the  subject's 
anxiety  response  would  be  a  facilitative  one.  Conversely,  the  less  control 
the  subject  felt,  the  more  debilitating  the  anxiety  response. 

In  a  study  of  university  undergraduates,  Johnson  and  Kilmann  (1975) 
found  that  internally  rated  students  rated  themselves  as  more  confident  of 
problem-solving  abilities  than  did  external  rated  students. 

User  Experience  and  the  Interface 

Martin's  (cited  in  Ambardar,  1984)  insistence  on  matching 
user-interface  characteristics  with  those  of  the  user  was  widely  accepted  in 
the  HCI  community.  As  Ambardar  (1984)  noted,  however,  "almost 
without  exception,  the  only  user  characteristic  that  has  been  systematically 
co-varied  with  interface  design  is  what  might  generally  be  termed 
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'training'  "  (p.  207).  This  work  has  led,  for  example,  to  the  well-known 
recommendation  that  "novices"  generally  required  "constrained"  designs 
such  as  menu-based  interfaces  while  "expert"  users  (i.e.,  trained)  require 
more  flexible  approaches  (Ambardar,  1984).  The  tendency  to  focus  on 
"expert"  behavior  has  been  criticized  by  Bannon  and  Bodker  (1991)  who 
caution  against  concentrating  on  aspects  of  people  that  seem  amenable  to  a 
computational  metaphor.  Bannon  and  Bodker  contend  that  these  aspects 
cannot  be  meaningfully  detached  from  real  situations  of  human  activity  to 
which  they  apply  "activity  theory." 

Bannon  and  Bodker' s  claim  not  withstanding,  several  studies  have 
involved  specific  claims  about  the  pertinence  of  previous  computer 
experience.  Draper  (1986)  noted  that  experienced  users  often  possessed 
special  skills:  "Their  expertise  lies,  not  in  their  having  learned  enough  to 
solve  any  problem  immediately,  but  in  their  having  become  skilled  in 
gathering  information  and  supplementing  what  they  know"  (p.  354). 

Shneiderman  (1987)  divides  user  knowledge  into  syntactic  and 
semantic  domains.  Syntactic  knowledge  contains  device-dependent  details. 
According  to  Shneiderman,  users  are  much  more  troubled  by  syntactic 
irregularities  in  a  system.  Semantic  knowledge  forms  an  hierarchical 
structure  of  computer  concepts  and  task  concepts.  Semantic  knowledge 
must  be  acquired  but  is  thought  to  be  fairly  stable  in  memory.  As  an 
example,  Shneiderman  cites  attributes  of  a  computer  file  such  as  size, 
name,  and  storage  location.  Although  these  concepts  were  once  known  to 
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only  a  small  number  of  computer  scientists,  they  are  now  taught  in 
elementary  school. 

Similarly,  Hammond  and  Barnard  (1984)  classify  knowledge  required 
in  system  learning  into  two  domains,  primary  and  secondary.  Primary 
knowledge  is  inferred  from  the  nature  of  the  demands  of  the  task  and  the 
system  and  is  required  for  successful  use  of  the  system.  Primary 
knowledge  consists  of  knowledge  of  the  computer  version  of  the  domain, 
knowledge  of  the  problem,  knowledge  of  system  operations,  knowledge  of 
physical  interface,  and  knowledge  of  interface  dialogue.  Secondary 
knowledge  sources  are  called  upon  to  infer  primary  knowledge  which  is 
lacking  or  uncertain.  Secondary  knowledge  (either  interference  or 
facilitation)  consists  of  knowledge  of  the  domain,  knowledge  of  the 
workbase  version  of  the  domain  (card  catalog,  etc.),  knowledge  of  natural 
language,  and  knowledge  of  other  machines  and  procedures. 


CHAPTER  3 
METHODOLOGY 

This  study  is  designed  to  determine  whether  proactive,  computerized 
guide-based  help  intervention  in  a  problem  space  can  be  more  effective 
than  on-demand  help  intervention  only  with  users  having  varying 
experience  and  varying  locus  of  control  indices.  Specifically,  the 
treatments  present  mediated  learning  experiences  (MLE),  alternatively 
offered  as  on-demand  guide  assistance  or  proactive,  system-initiated  guide 
assistance  to  participants  actively  involved  in  a  problem  space.  In  addition, 
the  study  seeks  to  determine  whether  proactive  guide-based  help  is 
perceived  by  the  user  as  being  more  useful  and  more  enjoyable. 

Research  Design 

A  split-plot  repeated  measures  design  was  used  in  the  study.  Each 
group  received  both  proactive  and  nonproactive  digital  video  guide 
assistance  during  problem-solving  activities.  Subjects  were  randomly 
assigned  to  receive  a  given  type  of  digital  video  guide  assistance  during  the 
first  problem  scenario.  During  the  second  problem  scenario  the  type  of 
assistance  each  subject  received  was  switched. 

Regression  analyses  were  performed  on  the  data  in  this  study. 
Depending  on  the  nature  of  the  particular  variables  used  in  an  analysis, 
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either  multiple  regression  or  logistic  regression  models  were  used.  When 
appropriate,  a  standard  analysis  of  variance  was  used.  An  analysis  of 
variance  (ANOVA)  partitions  the  total  variation  of  a  sample  into 
components  and  tests  that  the  means  are  equal.  The  components,  between 
group  variance  and  within  group  (error)  variance,  are  used  to  compute  an 
F  ratio  that  evaluates  the  effectiveness  of  the  model.  An  ANOVA  is  used 
when  the  measurement  level  of  the  variable  in  question  is  either  nominal  or 
ordinal  and  the  response  is  interval  (Sail,  Ng,  &  Hecht,  1991). 

The  use  of  multiple  regression  is  increasing  primarily  because  of  its 
versatility  and  precision  but  also  because  of  its  application  to  data 
representing  any  scale  of  measurement  (Gay,  1992).  In  standard 
regression  analysis,  the  mean  of  the  dependent  variable  is  modeled.  When 
the  dependent  variable  is  dichotomous,  the  proportion  of  responses  in  one 
of  the  categorical  variables  to  the  interval  independent  variables  is  modeled 
in  a  logistic  regression  (Agresti  &  Finlay,  1986),  that  is,  logistic  regression 
models  how  the  proportion  of  responses  in  one  of  two  categories  depends 
on  independent  variables.  When  calculating  the  effect  of  a  variable,  it  is 
necessary  to  calculate  the  probability  that  a  randomly  selected  subject 
chooses  one  of  the  dichotomous  responses.  Since  the  probability  (tc)  of 
such  an  event  is  linear  for  a  single  independent  variable,  it  is  possible  to 
generate  probabilities  below  0  or  above  1  for  extreme  values  of  X.  This 
condition  is  illustrated  in  Figure  3-1.  Rather  than  straight  lines,  logistic 
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regression  models  use  more  realistic  S-curves,  also  shown  in  Figure  3-1, 
thus  insuring  that  probabilities  always  fall  between  zero  and  one. 


Figure  3-1.  Linear  and  Logistic  Regression  Models  for  a  (0,1)  Response 
Source:  Agresti  &  Finlay  (1986,  p.  483). 

Nominal  logistic  regression  is  an  estimation  of  the  probability  of 
choosing  one  of  the  response  levels  as  a  smooth  function  of  the  factor. 
Probabilities  must  be  between  0  and  1  and  sum  to  1  across  response  levels 
for  a  given  factor  value  (Hosmer  and  Lemeshow,  1989).  The  fitting 
principal  for  nominal  logistic  regression  curves  estimates  maximum 
likelihood  of  fit  which  minimizes  the  sum  of  -log(Pi),  where  Pi  is  the 
probability  attributed  to  the  response  that  actually  occurred.  Therefore, 
-LogLikelihood  statistics  are  reported  for  logistic  regression  models. 
Analysis  of  LogLikelihood  tables  correspond  to  Analysis  of  Variance  tables 
for  interval  data,  and  -LogLikelihood  for  categorical  data  corresponds  to 
Sums  of  Squares  for  interval  data  (SAS  Institute,  1989).  Likelihood-Ratio 


A 


Logistic  (1) 
Logistic  (2) 


0 


41 


tests  examine  the  likelihood  that  all  parameters  that  make  up  an  effect  are 
zero  (Sail,  Ng,  &  Hecht,  1991). 

Additional  statistics  reported  for  the  regression  models  include  the 
Rsquare  statistic  which  is  the  proportion  of  total  uncertainty  which  is 
attributed  to  the  model  fit.  Rsquare  values  range  from  0,  which  means  that 
there  is  no  gain  by  using  the  model  over  fixed  background  response  rates, 
to  1 ,  which  indicates  a  perfect  fit.  High  Rsquare  statistics  are  rare  in 
categorical  models  (SAS  Institute,  1989).  The  chi-square  statistic  measures 
how  well  the  categorical  model  fits  the  data,  that  is,  it  is  a  measure  which 
tests  the  hypothesis  that  the  model  fits  no  better  than  fixed  response  rates 
across  the  whole  sample.  The  Prob>ChiSq  statistic,  often  referred  to  as  the 
p-statistic,  represents  the  probability  of  getting,  by  chance  alone,  a 
chi-square  value  greater  than  the  one  computed  for  the  model. 

Leverage  plots  are  used  to  illustrate  a  given  effect's  significance. 
Effects  tests  in  a  logistic  regression  compare  a  model  with  and  without  the 
effect  and  see  if  the  difference  in  the  Loglikelihood  is  significant. 
Similarly,  a  leverage  plot  compares  the  sum  of  squared  residuals  of  the 
model  with  the  effect  and  with  the  effect  removed.  If  residual  errors  are 
significantly  smaller  with  the  effect  included,  the  effect  has  significantly 
contributed  to  the  fit  (SAS  Institute,  1989).  Graphically,  the  components 
of  a  general  leverage  plot  are  given  in  Figure  3-2.  The  mean  line, 
represented  by  the  horizontal  line  in  the  figure,  represents  a  model  with  a 
constraint  to  zero  on  the  hypothesized  effect.  Leverage  plots  use  confidence 
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Figure  3-2.  General  Leverage  Plot 
Source:  SAS  Institute  (1989,  p.  323) 

curves  to  indicate  whether  or  not  the  test  being  run  is  significant.  If  the 
confidence  curves  cross  the  horizontal  line,  as  shown  in  Figure  3-3,  the 
model  is  significant. 


Fi  gure  3-3.  Confidence  curves  surrounding  line 
of  fit  in  a  significant  model 
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In  this  study,  predictors  for  the  analyses  related  to  the  win  outcome 
criterion,  that  is,  correct  solution,  were  computer  experience,  score  on  the 
Nowicki-Strickland  Locus  of  Control  inventory,  and  the  status  of 
proactive,  automatic  digital  video  guide  help.  Activity  functions  used  as 
predictors  related  to  win  outcome  included  amount  of  digital  video  help 
requested  and  the  amount  of  digital  video  help  received.  In  addition  to  the 
win  outcome  criterion,  frequency  of  video  help  requests,  preference 
toward  proactive  help,  perceived  help  usefulness,  and  enjoyment  of  the 
activity  were  each  held  as  criterion  (response)  variables.  Interactions 
between  predictor  variables  were  also  examined.  In  attitudinal  analyses, 
perceived  problem-solving  ability  and  perceived  problem  difficulty  level 
were  also  used  as  predictors. 

Population 

The  population  for  this  study  consisted  of  college  students  enrolled  in 
the  undergraduate  program  at  the  University  of  Florida. 

Sample 

The  sample  for  this  study  was  drawn  from  Introduction  to  Instructional 
Computing  Classes  during  three  terms  in  1993.  The  subjects  were  juniors 
and  seniors.  Prior  to  the  study,  the  students  were  asked  by  their  instructor 
if  they  were  willing  to  participate  in  a  study  related  to  computers  and 
learning.  Immediately  preceding  the  study,  an  explanation  of  the 
procedures  used  in  the  study  was  explained  to  the  students.  Students  were 
randomly  assigned  to  two  experimental  groups.  A  total  of  114  students 
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responded,  and  of  these  105  students  were  included  in  the  study.  Computer 
failures  and  improper  ID  matches  resulted  in  dropping  nine  partial  sets  of 
data. 

Instruments 

Three  instruments  were  used  in  this  study,  two  written  instruments  and 
one  computerized  instrument.  The  two  written  measures  were  the 
Nowicki-Strickland  Locus  of  Control  Inventory  for  Adults  and  an 
experience  measure  developed  for  the  purpose  of  this  study. 

The  Nowicki-Strickland  Locus  of  Control  Inventory  for  Adults  consists 
of  40  yes-no  questions.  The  adult  version  of  the  inventory  was  developed 
in  response  to  the  shortcomings  of  the  Rotter  (1966)  Locus  of  Control 
Scale  with  respect  to  college  and  noncollege  population  generalizability 
(Nowicki  &  Duke,  1974).  The  test  is  keyed  in  an  external  direction,  the 
total  score  being  the  number  of  external  responses  (Slayton,  1984).  Thus, 
the  higher  the  score,  the  more  an  external  locus  of  control  is  indicated. 
The  Nowicki  Strickland  test,  from  its  format  and  item  content,  represents 
expectancy  of  competence,  which  is  one  aspect  of  the  I-E  continuum 
(Slayton,  1984). 

The  experience  survey  was  created  by  the  experimenter  and  based  on  a 
model  used  in  an  earlier  computer-based  study.  The  survey  was  designed 
to  determine  a  level  of  familiarity  with  common  software  packages, 
research  facilities,  and  operating  systems  as  well  as  the  participants'  total 
previous  computer  use.  The  experience  survey  was  graded  according  to 
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the  following  criteria:  (a)  longevity  of  computer  use,  (b)  frequency  of 
computer  use,  (c)  type  of  activities  done  on  the  computer,  (d)  software 
package  familiarity,  and  (e)  operating  system  familiarity.  Participants 
were  identified  as  experienced  or  nonexperienced  according  to  a  point  scale 
rating.  The  survey  was  graded  (a  straightforward  counting  procedure)  by 
the  experimenter  and  two  other  people  identified  as  having  a  strong 
background  in  computing  and  instruction.  An  inter-scorer  reliability  of 
.9915  existed. 

In  addition  to  the  written  measures,  a  computer  program  based  on  the 
Coin  Challenge  game  developed  by  Dr.  Elroy  Bolduc  was  created  by  the 
researcher  to  determine  if  varying  the  type  of  help  intervention  given  the 
student  had  an  effect  on  game  performance.  As  with  Selz's  productive 
thinking  model,  the  activity  is  designed  to  give  the  user  what  Selz  called  a 
'partial  result'  which  must  be  examined  and  evaluated  before  a  new 
knowledge  complex  can  be  assembled.  Essentially,  Coin  Challenge  puts  the 
user  into  a  problem  space  in  which  deductive  logic  can  be  applied  for 
positive  gain  or  task  completion.  The  related  help,  then,  must  introduce 
the  user  to  this  metric.  The  help  component  deals  with  introducing  the 
user  to  the  functional  requirements  (i.e.,  rules  and  goals)  by  using  a 
preliminary  computerized  slide  presentation.  Subsequently,  the  digital 
video  guide  help  available  in  the  problem  space  assists  users  in  dealing  with 
problem  solving  as  well  as  functional  misques  or  ambiguities  (for  more  on 
this  approach  see  Martin,  1988). 
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The  computer  activity,  Coin  Challenge,  includes  a  problem  scenario 
and  two  five-question  surveys.  The  surveys  are  administered  after  the  first 
and  second  problems.  They  incorporate  a  semantic  differential  scale 
(Osgood,  Suci,  &  Tannenbaum,  1957),  and  subjects  select  values  along  the 
scale  by  clicking  the  mouse  on  the  appropriate  number.  The  questions  on 
the  survey  are  as  follows: 

How  would  you  classify  your  computer  expertise? 

1 .  Use  the  computer  for  word  processing  or  not  at  all. 

3.  Use  the  computer  once  in  a  while  for  various  things. 

5.  Use  the  computer  every  day.  Familiar  with  a  lot  of  software. 

How  would  you  rate  your  problem-solving  ability? 

1. 1  have  never  been  good  at  solving  logic  problems. 
3. 1  consider  myself  an  average  problem  solver. 
5. 1  enjoy  doing  problem  solving  activities. 

How  would  you  rate  the  game's  difficulty  level? 

1.  It  was  extremely  easy  to  learn  and  play. 
3.  It  was  fairly  difficult  to  learn  and  play. 
5.  It  was  very  difficult  to  learn  and  play. 

Were  the  game's  HELP  features  useful  to  you? 

1.  The  help  features  were  useless. 

3. 1  used  the  help  features  occasionally. 

5. 1  could  not  have  played  the  game  without  the  help  features. 

SET  TWO: 

In  one  game,  help  was  automatic.  In  the  other,  you  had  to  request 
help.  Which  do  you  prefer? 

1. 1  want  to  request  help.  I  hated  the  automatic  help! 

3. 1  don't  really  have  an  opinion,  it  doesn't  affect  the  game. 

5. 1  loved  being  given  hints  by  the  helper. 
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Did  you  enjoy  playing  the  game? 

1.  No,  I  didn't  like  this  game  at  all! 
3.  It  was  okay. 

5. 1  liked  this  game  a  great  deal. 

If  you  were  to  play  the  game  again,  which  type  of  help  would  you 
choose? 

1.  Automatic  Help:  I'd  like  the  helper  to  make  suggestions. 

2.  Requested  Help  Only:  I  want  to  ask  for  help  when  I  need  it. 

Would  you  like  to  play  again  now? 

1.  Yes,  I  would. 

2.  No  thanks. 


Coin  Challenge  software  was  designed  to  keep  an  audit  trace  of  all  user 
activity  in  the  problem  space.  From  this  trace,  specific  information  about 
the  activity  can  be  extracted.  In  fact,  the  entire  activity  can  be 
reconstructed  with  procedural  and  temporal  precision.  In  addition, 
responses  to  the  two  five-question  surveys  were  recorded,  but  not  analyzed, 
by  the  computer  program.  The  file  which  contained  the  audit  trace  and 
survey  information  for  each  participant  was  saved  and  collected  by  the 
experimenter  immediately  following  the  computer  session. 

Coin  Challenge  incorporates  a  digital  video  guide  which  is  central  to 
this  experiment.  The  guide  video  was  created  in  the  video  production 
studio  of  Norman  Hall  at  the  University  of  Rorida.  A  local  science 
instructor  played  the  part  of  the  guide.  The  researcher  wrote  a  script  for 
each  message  and  mediation  used  in  the  study.  The  video  (with  audio 
track)  was  recorded  using  1/2-half  inch  tape  and  then  transferred  to  VHS 
tape.  From  this  tape,  the  researcher  captured  specific  video  sequences 
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digitally  using  a  VideoSpigotT^^  capture  board  and  a  Macintosh^^  Quadra 
700  computer.  The  researcher  then  digitally  edited  the  digital  video  files. 
Video  sequences  were  captured  at  15  frames  per  second  in  256  color  mode. 
The  frame  size  for  capture  was  set  at  80  XI 60  pixels,  or  1/1 2th  of  a  640  X 
480  screen,  which  is  currently  a  de  facto  standard.  Finally,  the  researcher 
integrated  the  finished  video  segment  files  into  the  Coin  Challenge 
program. 

Stimuli 

The  problem-solving  activity  begins  with  a  linear  computerized  slide 
presentation  which  includes  the  objective  of  the  game,  the  directions,  the 
tools  available  to  the  user,  and  the  help  facilities  available.  Graphics, 
animation,  text,  and  sound  are  used  in  the  presentation.  The  user  may  not 
interrupt  the  presentation,  that  is,  each  subject  views  the  same  instructions. 

Upon  completion  of  the  instructions,  the  subject  is  asked  for  an  I.D. 
number  and  the  problem-solving  activity  begins.  The  user  is  presented 
with  a  screen  or  game  board  which  has  the  following  elements:  a  blank 
board  on  which  to  place  coins,  a  change  bank  which  is  the  source  of  coins, 
a  reminder  box  which  explains  the  meaning  of  the  feedback  mechanism 
(clues),  an  answer  cover,  and  a  Help  Box.  The  game  begins  with  a  video 
guide  appearing  in  the  Help  Box  which  says  "Click  Here  for  Help."  A 
static  picture  of  the  guide  then  remains  in  the  Help  Box.  In  this  study,  the 
guide  is  a  male  figure  wearing  a  bow  tie  and  glasses,  similar  to  the  classic 
computerized  agent  "Phil"  (Laurel,  1990). 
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The  object  of  the  game  is  to  try  to  determine  which  four  coins  are 
hidden  under  the  answers  cover  and  their  placement.  In  order  to  make  this 
determination,  the  user  is  expected  first  to  make  a  four-coin  guess.  This 
system  then  provides  feedback  consisting  of  how  many  coins  are  in  exactly 
the  correct  position,  how  many  coins  are  correct  but  in  the  wrong  position, 
and  by  how  many  cents  the  guess  is  off.  This  feedback  is  given  via 
indicator  codes.  Once  the  user  has  interpreted  and  evaluated  this  partial 
result,  another  more  educated  "guess"  is  made  and  evaluated  by  the  system. 
A  game  consists  of  up  to  six  such  guesses/attempts  or  a  win,  whichever 
comes  first. 

Coins  are  randomly  placed  under  the  answers  cover  by  the  computer 
from  a  pool  of  Half-DoUar,  Quarter,  Dime,  Nickel,  Penny,  and  Blank 
Token.  Rules  of  the  game  restrict  the  total  value  of  the  coins  under  the 
answers  cover  to  99  cents  or  less.  Thus,  the  game  resembles  the  Parker 
Brothers  game  MastermindTM  with  the  added  dimension  of  coin  value 
which  introduces  arithmetic  into  the  game. 

The  help  facilities  in  the  program  attempt  to  address  the  user's  needs  in 
several  ways.  Problem  solving  should  be  treated  as  a  process  (Kantowski 
1980;  Norman  1983;  Polya  1962),  and  the  help  facihties  of  the  program 
have  been  designed  to  provide  informational,  procedural,  and  strategic 
assistance.  Coin  Challenge  is  interactive  and  makes  algorithmically 
intelligent  decisions  regarding  the  needs  of  the  user.  For  example,  early  in 
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the  game,  or  if  the  user  takes  inappropriate  actions,  the  program  will  issue 
procedural  help  rather  than  strategic  help. 

Two  versions  of  the  Coin  Challenge  were  used  with  each  subject.  In 
one  version,  video  guide  help  is  automatic  or  proactive,  occurring  after 
specific  periods  of  user  inactivity,  or  when  help  is  requested  by  the  user. 
In  the  other,  help  must  be  requested  by  the  user.  This  is  accomplished  by 
moving  the  mouse  pointer  to  the  help  box  and  clicking  the  mouse  button. 
The  help  messages  given  are  the  same  in  the  two  versions. 

The  distinctions  in  the  stimuli  for  this  experiment  are  thus  centered 
around  the  nature  of  the  help  given  to  the  user.  First,  it  is  necessary  to 
offer  assistance  in  some  form  to  the  user,  and  requested  help  is  used  as  the 
baseline  for  this  help.  Second,  the  stimuli  had  to  incorporate  proactive  or 
automatic  help  in  order  to  measure  its  effect  on  the  user. 

Procedure 

The  written  measures  were  administered  in  a  supervised  classroom 
setting  to  groups  of  approximately  15  participants.  When  the  group 
completed  the  forms,  they  were  ushered  to  a  computer  lab  which  had  been 
set  up  with  the  Coin  Challenge  activity.  The  activity  was  created  for 
Macintosh  computers  using  HyperCard^M  from  Apple  Computer.  The 
laboratory  was  outfitted  with  Macintosh  LCII  computers,  13-inch  Apple 
Color  Monitors  (640  by  480  resolution)  with  stereo  headphones  attached  to 
each  station.  Every  participant  observed  a  computerized  slide  show  of 
instructions  which  lasted  approximately  3  minutes.  The  instructions 
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appeared  on  each  participants  monitor  and  were  noninterruptable.  The 
instructions  contained  graphics,  text,  animation,  and  sound.  After  viewing 
the  slide  show,  the  participant  was  asked  to  enter  the  same  I.D.  used  for  the 
written  dimension  of  the  study.  The  Coin  Challenge  activity  then  began. 

Each  participant  was  expected  to  do  the  activity  twice,  with  a  third 
game  optional.  One-half  of  the  participants  received  proactive  help  during 
the  first  game,  while  the  other  half  received  help  only  when  they  requested 
it.  After  the  first  game  the  participants  were  presented  with  an  informal 
attitudinal  survey  composed  of  five  questions  (see  Appendix  C).  A 
semantic  differential  scale  (Osgood,  Soci,  &  Tannenbaum,  1957)  format 
was  used  for  each  question.  Responses  were  made  by  simply  clicking  the 
mouse  on  the  appropriate  mark  on  the  scale. 

After  responding  to  the  five  questions,  the  Coin  Challenge  was 
reentered.  During  this  second  game,  the  type  of  help  was  reversed.  If 
proactive  help  was  in  effect  during  the  first  game,  it  was  disabled  so  that 
users  were  required  to  request  help  in  the  second.  Similarly,  if  it  were 
necessary  for  the  participant  to  request  help  in  the  first  game,  help  was 
automatic  in  the  second  game. 

After  the  second  game,  the  participants  were  again  asked  to  answer  a 
five-question  survey.  The  last  question  gives  them  the  option  of  entering 
the  activity  a  third  and  final  time.  The  experimenter  was  present  during 
the  computer  session  to  assist  with  technical  difficulties,  but  no  activity- 
related  feedback  was  given  to  the  participants  by  the  experimenter.  The 
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experimenter  also  took  notes  on  comments  participants  made  during  the 
activity. 

Summary 

The  present  study  was  designed  to  determine  the  effectiveness  of 
proactive  digital  video  guide-based  assistance  with  users  having  varying 
experience  and  varying  locus  of  control  indices.  The  type  of  digital  video 
guide  assistance  was  varied  to  determine  if  proactive,  automatic  help  affects 
a  user's  performance  in  a  novel  problem  space.  Although  the  frequency  of 
help  received  varied  depending  on  help  status,  the  help  received  in  both 
cases  was  the  same,  and  context  sensitive.  A  split-plot  design  was  used  in 
the  study. 

The  subjects  in  this  study  were  105  juniors  and  seniors  from  the 
University  of  Florida  who  were  enrolled  in  introductory  instructional 
computing  courses.  Each  subject  was  given  an  experience  survey,  a 
Nowicki-Strickland  Locus  of  Control  Inventory,  and  then  participated  in  a 
computerized  problem-solving  activity.  The  problem- solving  activity 
consisted  of  two  interactive  sessions  in  Coin  Challenge,  a  logic-  and 
arithmetic-based  problem  scenario,  and  a  computerized  attitude  survey 
with  questions  being  administered  before  and  after  the  second  Coin 
Challenge  session.  All  subjects'  actions  in  the  computer  portion  of  the 
study  were  recorded  by  the  computer  and  saved  for  subsequent  analyses. 


CHAPTER  4 
ANALYSIS  OF  THE  DATA 

This  purpose  of  this  study  was  to  determine  whether  proactive, 
computerized  guide-based  help  intervention  in  a  problem  space  is  more 
effective  than  on-demand  help  intervention  alone  with  users  classified  as 
possessing  a  range  of  locus-of-control  measures  and  experience. 
Specifically,  the  treatments  present  mediated  learning  experiences 
(MLE),  alternatively  offered  as  on-demand  guide  assistance  or 
proactive,  system-initiated  guide  assistance  to  participants  actively 
involved  in  the  problem  space.  In  addition,  the  study  seeks  to  determine 
what  factors  contribute  to  the  user's  perception  of  the  video  guide-based 
help. 

Results 

Multiple  regression,  logistic  regression  and  analysis  of  variance 
techniques  were  used  to  analyze  the  data.  Each  of  the  hypotheses  is  stated 
in  operational  null  form  with  the  relevant  statistics  presented  in  summary 
tables.  Graphs  and  leverage  charts  have  been  included  for  significant 
effects. 
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Hypothesis  1:  There  is  no  significant  difference  in 
achievement  when  the  amount  of  video  guide-based  help 
intervention  is  varied. 
The  logistic  regression  performed  on  the  data  yielded  a  p-statistic 
(Prob>Chi  Square)  of  0.0045  for  the  main  effect  of  amount  of  video  guide- 
based  help  intervention.  Since  this  is  significant  at  the  p<  .01  level,  the  null 
hypothesis  was  rejected.  The  summary  of  fit  and  parameter  estimates  are 
given  in  Table  4- 1  and  a  graph  of  expected  effects  of  total  help  received  on 
win  outcome  is  shown  in  Figure  4-1. 

Table  4-1 

The  Effects  of  Total  Help  Received  on  Win  Outcome 


Rsquare  (U)  0.078945 
Observations  (or  Sum  Wgts)  105 


Analysis  of  LogLikelihood 

Source 

DF 

-LogLikelihood  ChiSquare 

Model 

1 

5.736267 

11.47253 

Error 

103 

66.925094 

Prob>ChiSq 

C  Total 

104 

72.661361 

0.000706 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

ChiSquare  Prob>ChiSq 

Intercept 

-0.5487473 

0.2858245 

3.69  0.0549 

Help  Freq 

0.07280954 

0.0256197 

8.08  0.0045 
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Help  Freq 

Figure  4-1.  The  Effects  of  Total  Help  Received  on  Win 
Outcome 

Hypothesis  2:  There  is  no  significant  difference  in 
achievement  when  the  frequency  of  user  requests  for  help 
varies. 

Logistic  regression  analyses  were  performed  on  the  data.  The  model 
(Table  4-2)  includes  relevant  data  to  test  the  hypotheses  related  to  subject 
parameters  and  activity  concerning  locus  of  control  orientation,  level  of 
experience,  help  type  (manual  or  proactive),  and  frequency  of  requested 
help.  A  preliminary  analysis  yielded  nonsignificant  results  for  all 
interaction  effects.  The  analysis  of  Log  Likehhood  which  tests  the  model  is 
shown  in  Table  4-2.  The  analysis  resulted  in  a  p-statistic  of  0.002436 
which  is  significant  at  the  p<.01  level.  However,  the  model  exhibited  a 
significant  lack  of  fit  (p=0.023001).  A  stepwise  logistic  regression  was 
performed  eliminating  the  locus  of  control  variable  which  had  produced  an 


Table  4-2 


The  Effects  of  Locus  of  Control,  Experience.  Help  Status,  and  Requested 
Help  Frequency  on  Win  Outcome 


Rsquare  (U) 

0.113419 

Observations  (or  Sum  Wgts) 

105 

Analysis  of  LogLikelihood 

Source 

DF 

-LogLikelihood 

ChiSquare 

Model 

4 

8.241173 

16.48235 

Error 

100 

64.420188 

Prob>ChiSq 

C  Total 

104 

72.661361 

0.002436 

Lack  of  Fit 

Source 

DF 

-LogLikelihood 

ChiSquare 

Lack  of  Fit 

94 

61.647599 

123.2952 

Pure  Error 

6 

2.772589 

Prob>ChiSq 

Total  Error 

100 

64.420188 

0.023001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

ChiSquare  Prob>ChiSq 

Intercept 

-1.4742017 

0.7163256 

4.24 

0.0396 

LOG 

0.02606149 

0.051617 

0.25 

0.6136 

Experience 

0.06618216 

0.0423318 

2.44 

0.1180 

Proactiv[0-1] 

-0.4874518 

0.2329204 

4.38 

0.0364 

Requested  Help 

0.41296946 

0.1422057 

8.43 

0.0037 
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insignificant  effect.  The  resulting  model  is  shown  in  Table  4-3.  The 
model  was  based  on  the  frequency  of  help  requested  during  the  activity, 
experience,  and  proactive  help  status  and  yielded  a  p-statistic  of  0.001019 
which  is  significant  at  the  p<.01  level.  The  main  effect  test  for  help 
requested  by  the  user,  indicated  by  Requested  Help  in  Table  4-3,  was  based 
on  the  frequency  of  help  requests  during  the  activity  and  yielded  a 
p-statistic  of  0.0039  which  is  significant  at  the  p<  .01  level.  Therefore,  the 
null  hypothesis  was  rejected.  Figure  4-2  illustrates  the  expected  effect  of 
requested  help  on  win  status  based  on  the  model  in  Table  4-3. 

Table  4-3 


The  Effects  of  Experience.  Proactive  Help  Status  and  Requested  Help 
Frequency  on  Win  Outcome 


Rsquare  (U) 

0.111656 

Observations  (or  Sum  Wgts) 

105 

Analysis  of  LogLikelihood 

Source 

DF 

-LogLikelihood 

ChiSquare 

Model 

3 

8.113044 

16.22609 

Error 

101 

64.548317 

Prob>ChiSq 

C  Total 

104 

72.661361 

0.001019 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

ChiSquare  Prob>ChiSq 

Intercept 

-1.2289784 

0.5214269 

5.56 

0.0184 

Experience 

0.06495216 

0.0423236 

2.36 

0.1249 

Proactive[0-1] 

-0.4754479 

0.2311267 

4.23 

0.0397 

Requested  Help 

0.40914646 

0.1417484 

8.33 

0.0039 
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-1      123456789  10 
Requested  Help 

Figure  4-2.  The  Effect  of  Requested  Help  Frequency  on  Win 
Outcome 

Hypothesis  3:  There  is  no  significant  difference  in  the  number  of 
requests  for  help  when  the  type  (proactive  or  manual)  of  digital 
video  guide  assistance  is  varied. 
The  analysis  of  variance  performed  on  the  data  resulted  in  an  F  statistic 
of  6.9021  (Table  4-4).  Because  this  was  significant  at  the  p<  .01  level,  the 
null  hypothesis  was  rejected.  A  leverage  plot  representing  the  contribution 
of  proactive  help  status  on  frequency  of  help  requests  is  shown  in  Figure 
4-3. 


Table  4-4 

The  Effect  of  Proactive  Help  Status  on  Frequency  of  Help  Requests 


Rsquare 

Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 


0.062802 
2.262601 
1.685714 
105 


Analysis  of  Variance 
Source         DP       Sum  of  Squares 
Model  1  35.33413 

Error  103  527.29444 
C  Total        104  562.62857 


Mean  Square 
35.33416.9021 
5.1194Prob>F 
0.0099 


F  Ratio 
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Figure  4-3.  Leverage  Chart  for  the  Effect  of  Proactive  Help  Status  on 
Requests  for  Help 
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Hypothesis  4:  There  is  no  significant  difference  in 
achievement  between  students  with  varying  locus  of 
control  indices  when  the  type  of  video  guide-based  help 
intervention  is  varied. 
The  logistic  regression  analysis  performed  (Table  4-2)  yielded  a 

p-statistic  which  was  not  significant  at  the  p<  .05  level.  Therefore,  this  null 

hypothesis  could  not  be  rejected. 

Hypothesis  5:  There  is  no  significant  difference  in  student 
achievement  due  to  an  interaction  effect  of  locus  of 
control  orientation  and  the  type  of  video  guide-based  help 
intervention. 

The  logistic  regression  (Table  4-5)  performed  show  no  significant 
interaction  between  the  locus-of-control  orientation  and  the  type  of  video 
guide-based  help.  Therefore,  this  null  hypothesis  was  not  rejected. 
Hypothesis  6:  There  is  no  significant  difference  in 
achievement  between  students  with  varying  experience 
when  the  type  of  video  guide-based  help  intervention  is 
varied. 

Because  the  regression  performed  on  the  data  yielded  p-statistics 
for  the  experience  and  help  status  effects  which  were  not  significant 
at  the  p<  .05  level  (Table  4-3),  this  null  hypothesis  could  not  be 
rejected. 
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Table  4-5 

The  Interaction  Effects  between  Locus  of  Control.  Experience,  and 
Requested  Help  on  Win  Outcome 


Rsquare  (U)  0.117596 
Observations  (or  Sum  Wgts)  105 


Analysis  of  LogLikelihood 

Source  DF  -LogLikelihood  ChiSquare 

Model  7  8.544710  17.08942 

Error  97  64.116652  Prob>ChiSq 

C  Total  104  72.661361  0.016829 


Parameter  Estimates 

Term  Estimate  Std  Error 

Intercept  -1.2957163  1.2607884 

LOC*Experience  0.00217536  0.0117674 

LOC*Proactiv[0-l]0.01 194278  0.054916 

Exp.  *Proactiv[0- 1  ]  0.0323670 1  0.0434285 


ChiSquare  Prob>ChiSq 

1.06  0.3041 

0.03  0.8533 

0.05  0.8278 

0.56  0.4561 


Hypothesis  7:  There  is  no  significant  difference  in 
achievement  due  to  an  interaction  effect  of  type  of  video 
guide-based  help  intervention  and  experience. 
The  p-statistic  for  the  interaction  between  video  guide-based  help 
intervention  and  experience  obtained  by  performing  a  logistic  regression 
on  the  data  (Table  4-5)  was  not  significant  at  the  p<  .05  level. 
Consequently,  this  null  hypothesis  was  not  rejected. 
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Hypothesis  8:  There  is  no  significant  difference  in 
achievement  when  the  type  of  video  guide-based  help 
intervention  is  varied  due  to  an  interaction  effect  of  locus  of 
control  orientation  and  experience. 
The  logistic  regression  analysis  performed  on  the  data  (shown  in  Table 
4-5)  yielded  no  significant  results  for  the  interaction  effect  between  locus 
of  control  orientation  and  experience.  Therefore,  the  null  hypothesis  was 
not  rejected. 

Hypothesis  9:  There  is  no  significant  difference  in  student 
preference  towards  proactive  help  due  to  an  estimation  of  their 
own  problem-solving  ability. 
The  F  ratio  for  problem-solving  ability  was  5.0153  at  df=l  (Table 
4-6).  Because  an  F  ratio  of  5.0153  is  significant  at  the  p<.05  level,  this 
null  hypothesis  was  rejected.  The  leverage  plot  showing  the  contribution 
of  perceived  problem- solving  ability  on  preference  toward  proactive 
digital  video  guide  help  is  given  in  Figure  4-4. 

Hypothesis  10:  There  is  no  significant  difference  in 
student  preference  towards  proactive  video  guide  help 
due  to  their  estimation  of  the  difficulty  of  problems 
presented  to  them. 

The  computed  F  ratio  of  0.001 1  (Table  4-6)  is  not  significant  at  the  p< 
.05  level.  This  null  hypothesis  was  not  rejected. 
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Table  4-6 

The  Effect  of  Perceived  Problem  Solving  Ability  on  Preference  Toward 
Proactive  Digital  Video  Guide  Help 


Rsquare 

Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 


0.0485 
1.153474 
2.114286 
105 


Parameter  Estimates 

Term                Estimate  Std  Error  t  Ratio  Prob>ltl 

Intercept             2.8141897  0.47837  5.88  0.0000 

P.Ablity             -0.230607  0.10297  -2.24  0.0273 

DiffLev              0.004151  0.12673  0.03  0.9739 


Source  Nparm  DP  Sum  of  Squares  F  Ratio  Prob>F 
P.Ablity  1     1     6.6728022  5.0153  0.0273 

DiffLev  1     1     0.0014274  0.0011  0.9739 


Figure  4-4.  Effect  of  Perceived  Problem  Solving  Ability  on  Preference 
Toward  Proactive  Digital  Video  Guide  Help 
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Hypothesis  11:  There  is  no  significant  difference  in  student 
preference  towards  proactive  video  guide  help  due  to  an 
interaction  between  an  estimation  of  their  own  problem- 
solving  ability  and  their  estimation  of  the  difficulty  of 
problems  presented  to  them. 
An  analysis  of  variance  was  performed  on  the  data.  The  F  ratio  for 
the  interaction  between  estimation  of  problem-solving  ability  and 
estimation  of  problem  difficulty  was  0.0055  at  df=l  is  not  significant 
at  the  p<  .05  level.  Consequently,  this  null  hypothesis  was  not 
rejected. 

Hypothesis  12:  There  is  no  significant  difference  in 
student  preference  towards  type  of  video  guide  due  to  the 
effects  of  their  locus-of-control  orientation,  experience, 
and/or  the  interaction  between  locus  of  control  and 
experience. 

An  analysis  of  variance  was  performed  on  the  various  effects  in  this 
model  which  examined  preference  toward  proactive  help.  Significant 
effects  at  the  p<  .05  level  were  found  for  locus  of  control,  experience,  and 
the  interaction  between  locus  of  control  and  experience  (Table  4-7). 
However,  since  the  model  itself  is  not  significant  at  the  p<  .05  level,  the 
null  hypothesis  could  not  be  rejected. 
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Table  4-  7 

The  Effect  of  Locus  of  Control.  Experience,  and  the  Interaction  Between 
Locus  of  Control  and  Experience  on  Preference  Towards  Type  of  Video 
Guide-based  Help  Intervention 


Rsquare  0.070314 

Root  Mean  Square  Error  1.145805 

Mean  of  Response  2. 1 14286 

Observations  (or  Sum  Wgts)  105 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>ltl 

Intercept 

0.5700636 

0.5792 

0.98 

0.3274 

LOC 

0.179198 

0.06644 

2.70 

0.0082 

Exp. 

0.1102617 

0.04664 

2.36 

0.0200 

LOC*Exp. 

-0.013017 

0.0056 

-2.32 

0.0221 

Analysis  of  Variance 

Source         jDF       Sum  of  Squares  Mean  Square  F  Ratio 

Model           3         10.02873  3.34291  2.5463 

Error         101        132.59984  1.31287  Prob>F 

C  Total        104        142.62857  0.0602 


Hvpothesis  13:  There  is  no  significant  difference  in  students' 
expressed  enjoyment  of  the  problem-solving  activity  due  to 
the  effects  of  locus-of-control  orientation,  experience, 
perception  of  their  own  problem-solving  ability,  perception 
of  the  problem's  difficulty  and/or  the  delivery  system's 
proactive  help  status. 
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A  multiple  regression  was  performed  on  the  data  which  examined  the 
effects  of  several  parameters  related  to  the  interaction  scenario  (Table  4-8). 
A  significant  effect  at  the  p<  .05  level  was  found  for  locus-of-control 
orientation  with  an  F  ratio  of  4.1276.  The  estimation  of  problem- solving 
ability  also  yielded  a  significant  effect  (p<  .01)  with  a  computed  F  ratio  of 
1 1.0524.  Perceived  difficulty  level  was  also  significant  (p<  .01)  with  an  F 
ratio  of  1 1.4151.  In  addition,  the  interaction  between  locus  of  control 
orientation  and  level  of  experience  was  significant  at  the  p<.05  level. 
However,  since  the  level  of  experience  produced  an  F  ratio  of  1.76  and  was 
not  significant  at  p<.05,  this  null  hypothesis  was  accepted. 

Hypothesis  14:  There  is  no  significant  difference  in  students' 
perceived  usefulness  of  video  guide  help  due  to  the  effects  of 
locus-of-control  orientation,  experience,  perception  of  their 
own  problem- solving  ability,  perception  of  the  problem's 
difficulty  and/or  the  delivery  system's  proactive  help  status. 
The  multiple  regression  analysis  performed  on  this  model  shows  a 
significant  effect  for  the  main  effects  locus-of-control  orientation  and 
perceived  problem-solving  ability  and  for  the  interaction  effects  between 
locus  of  control  and  experience  and  the  interaction  between  perceived 
problem-solving  ability  and  perceived  difficulty  level.  (Table  4-9).  All 
other  main  effects  and  interactions  were  not  significant  at  the  p<  .05  level. 
Consequently,  this  null  hypothesis  was  not  rejected. 
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Table  4-8 

The  Effect  of  Locus  of  Control.  Experience.  Perceived  Problem  Solving 
Capability.  Perceived  Problem  Difficulty  Level,  and  Related  Interactions 
on  Enjoyment  of  the  Activity 


Rsquare  0.235634 

Root  Mean  Square  Error  0.920375 

Mean  of  Response  3.42857 1 

Observations  (or  Sum  Wgts)  105 


Whole-Model  Test 
Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

5 

25.85236 

5.17047 

6.1038 

Error 

99 

83.86192 

0.84709 

Prob>F 

C  Total 

104 

109.71429 

0.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error       t  Ratio 

Prob>ltl 

Intercept 

2.5675015 

0.6124 

4.19 

0.0001 

LOC 

0.1093577 

0.05383 

2.03 

0.0449 

Exp. 

0.0661365 

0.03749 

1.76 

0.0808 

LOC*Exp. 

-0.009769 

0.00452 

-2.16 

0.0329 

P.Ablity 

0.2813608 

0.08463 

3.32 

0.0012 

DiffLev 

-0.342581 

0.1014 

-3.38 

0.0010 

Effect  Test 


Source         Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob>F 

LOC                  1  1 

3.4964652 

4.1276 

0.0449 

Exp.                   1  1 

2.6360329 

3.1119 

0.0808 

LOC*Exp.           1  1 

3.9651380 

4.6809 

0.0329 

P.Ablity              1  1 

9.3623821 

11.0524 

0.0012 

DiffLev               1  1 

9.6696095 

11.4151 

0.0010 
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Figure  4-5.  Factors  Contributing  to  Enjoyment  of  the  Problem- 
Solving  Activity 

a)  Leverage  Plot:  The  Effect  of  Locus  of  Control  on  Activity 
Enjoyment 

b)  Leverage  Plot:  The  Effect  of  the  Interaction  Between  Locus 
of  Control  and  Experience  on  Activity  Enjoyment 

c)  Leverage  Plot:  The  Effect  of  Perceived  Problem  Solving 
Ability  on  Activity  Enjoyment 

d)  Leverage  Plot:  The  Effect  of  Perceived  Difficulty  Level  on 
Activity  Enjoyment 
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Table  4-9 

The  Effect  of  Locus  of  Control.  Experience.  Perceived  Problem-Solving 
Capability.  Perceived  Problem  Difficulty  Level,  and  Related  Interactions 
on  Perceived  Usefulness  of  Digital  Video  Guide  Help 


Rsquare 

Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 


0.173846 
0.971522 
2.352381 
105 


Analysis  of  Variance 
Source         DF       Sum  of  Squares 
Model  6  19.46418 

Error  98  92.49772 

C  Total       104  111.96190 


Mean  Square 
3.24403 
0.94385 


F  Ratio 
3.4370 
Prob>F 
0.0040 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>ltl 

Intercept 

2.3074678 

0.83825 

2.75 

0.0070 

LOC 

0.1619433 

0.05692 

2.85 

0.0054 

Exp. 

0.0749348 

0.03963 

1.89 

0.0616 

LOC*Exp. 

-0.011377 

0.00477 

-2.38 

0.0191 

P.Ablity 

-0.512272 

0.19774 

-2.59 

0.0110 

DiffLev 

-0.457606 

0.26431 

-1.73 

0.0865 

P.Ablity*DiffLev 

0.2166131 

0.08611 

2.52 

0.0135 
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Figure  4-6.  Factors  Contributing  to  Perceived  Usefulness  of  Digital  Video 
Guide-based  Help 

a)  Leverage  Plot:  The  Effect  of  Locus  of  Control  on 
Perceived  Usefulness  of  Digital  Video  Guide  Help 

b)  Leverage  Plot:  The  Effect  of  the  Interaction  Between  Locus 
of  Control  and  Experience  on  Perceived  Usefulness  of 
Digital  Video  Guide  Help 

c)  Leverage  Plot:  The  Effect  of  Perceived  Problem  Solving 
Ability  on  Perceived  Usefulness  of  Digital  Video  Guide 
Help 

d)  Leverage  Plot:  The  Effect  of  the  Interaction  Between 
Perceived  Difficulty  Level  and  Perceived  Difficulty  Level 
on  Perceived  Usefulness  of  Digital  Video  Guide  Help 
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Summary 

The  research  questions  addressed  by  this  study  fall  into  three  categories 
designed  to  investigate  the  somewhat  broader  questions: 

1.  Does  digital  video  guide-based  help  intervention  support  a  user's 
efforts  in  a  problem  space? 

2.  Is  the  effectiveness  of  proactive  digital  video  guide-based  help 
affected  by  parameters  such  as  locus  of  control  and  experience? 

3.  What  factors  contribute  to  the  user's  perception  of  the  video  guide- 
based  help  system? 

The  analyses  of  the  data  indicate  that  it  is  likely  that  more  digital  video 
guide-based  help  will  lead  to  more  success  in  solving  the  problem 
correctly,  and  requesting  such  digital  video  guide-based  assistance  will  also 
have  a  positive  effect  on  problem-solving  ability.  Experience  and  locus  of 
control  are  not  significantly  related  to  the  problem- solving  achievement 
when  the  type  of  digital  video  help  is  varied  and  neither  is  the  interaction 
between  experience  and  locus  of  control. 

Students  who  perceive  themselves  as  good  problem  solvers  are  less 
likely  to  prefer  proactive,  automatic  digital  video  help.  Similarly,  students 
with  a  higher  locus  of  control  index  and  those  who  consider  themselves 
good  problem  solvers  enjoyed  the  game  to  a  significantly  greater  extent. 
As  might  be  expected,  the  more  difficult  the  problem  is  perceived  to  be, 
the  less  likely  the  student  is  to  enjoy  the  activity.  Students  who  were  rated 
as  more  external  in  locus-of-control  orientation  perceived  the  digital  video 
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guide  as  more  helpful.  Students  considering  themselves  good  problem 
solvers  found  digital  video  guide  assistance  less  useful.  There  was  also  a 
significant  interaction  between  locus  of  control  and  experience  in  the 
subjects'  perception  of  the  usefulness  of  digital  video  guide  assistance. 


CHAPTER  5 
DISCUSSION  AND  CONCLUSION 

Introduction 

The  research  questions  addressed  by  this  study  were  designed  to 
examine  various  aspects  of  rapidly  growing  concern  in  the  area  of 
human-computer  interaction.  The  study  focuses  on  a  digital  video  guide 
which  is  used  to  offer  advice  to  users  actively  involved  in  a  problem  space. 
Using  guides  to  assist  users  in  computer-related  activities  is  a  relatively 
recent  phenomenon.  This  study,  therefore,  examined  the  effects  of  its 
principal  foci,  user  characteristics  related  to  experience  and  locus  of 
control,  and  also  surveyed  users  as  to  their  preferences  toward  the  help 
system  incorporated  in  the  study. 

The  study  investigated  the  broad  questions: 

1.  Does  digital  video  guide-based  help  intervention  support  a  user's 
efforts  in  a  problem  space? 

2.  Is  the  effectiveness  of  proactive  digital  video  guide-based  help 
affected  by  parameters  such  as  locus  of  control  and  experience? 

3.  What  factors  contribute  to  the  user's  perception  of  the  video  guide- 
based  help  system?  The  first  question  deals  with  the  appropriateness  of  a 
digital  video  guide  representation  as  a  help  facility.  Norman  (1993)  writes, 
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"Bad  representations  turn  problems  into  reflective  challenges.  Good 
representations  can  often  transform  the  same  problems  into  easy 
experiential  tasks"  (p.  55).  The  second  question  relates  to  the  digital  video 
guide's  appropriateness  for  the  person,  that  is,  does  it  enhance  the  person's 
ability  to  interpret  the  problem.  Specifically,  this  study  examined  whether 
user  characteristics  such  as  locus  of  control  index  and  experience  related  to 
success  in  the  problem  space  given  a  particular  type  of  digital  video  help. 
Finally,  the  third  question  is  directed  toward  the  appropriateness  of  the 
proactive  digital  video  guide  to  the  task  at  hand.  Did  the  users  perceive 
beneficial  or  relevant  consequences  to  using  the  help  facilities  in  the 
problem  space  based  on  the  proactive  status  of  the  digital  video  system 
help? 

Findings 

Hypothesis  1:  There  is  no  significant  difference  in 
achievement  when  the  amount  of  video  guide-based  help 
intervention  is  varied. 
The  null  hypothesis  was  rejected. 

Hypothesis  2:  There  is  no  significant  difference  in 
achievement  when  the  frequency  of  user  requests  for  help 
varies. 

The  null  hypothesis  was  rejected. 
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Hypothesis  3:  There  is  no  significant  difference  in  the  number 
of  requests  for  help  when  the  type  (proactive  or  manual)  of 
digital  video  guide  assistance  is  Vciried. 

The  null  hypothesis  was  rejected. 

Hypothesis  4:  There  is  no  significant  difference  in  achievement 
between  students  with  varying  locus  of  control  indices  when 
the  type  of  video  guide-based  help  intervention  is  varied. 

The  null  hypothesis  was  not  rejected. 

Hypothesis  5:  There  is  no  significant  difference  in  student 
achievement  due  to  an  interaction  effect  of  locus-of-control 
orientation  and  the  type  of  video  guide-based  help 
intervention. 

The  null  hypothesis  was  not  rejected. 

Hypothesis  6:  There  is  no  significant  difference  in  achievement 
between  students  with  varying  experience  when  the  type  of 
video  guide-based  help  intervention  is  varied. 

The  null  hypothesis  was  not  rejected. 

Hvpothesis  7:  There  is  no  significant  difference  in  achievement 
due  to  an  interaction  effect  of  type  of  video  guide-based  help 
intervention  and  experience. 

The  null  hypothesis  was  not  rejected. 

Hypothesis  8:  There  is  no  significant  difference  in  achievement 
when  the  type  of  video  guide-based  help  intervention  is  varied 


due  to  an  interaction  effect  of  locus  of  control  orientation  and 
experience. 

The  null  hypothesis  was  not  rejected. 

Hypothesis  9:  There  is  no  significant  difference  in  student 
preference  towards  proactive  help  due  to  an  estimation  of 
their  own  problem-solving  ability. 

The  null  hypothesis  was  rejected. 

Hvpothesis  10:  There  is  no  significant  difference  in  student 
preference  towards  proactive  video  guide  help  due  to  their 
estimation  of  the  difficulty  of  problems  presented  to  them. 

The  null  hypothesis  was  not  rejected. 

Hvpothesis  11:  There  is  no  significant  difference  in  student 
preference  towards  proactive  video  guide  help  due  to  an 
interaction  between  an  estimation  of  their  own  problem- 
solving  ability  and  their  estimation  of  the  difficulty  of 
problems  presented  to  them. 

The  null  hypothesis  was  not  rejected. 

Hvpothesis  12:  There  is  no  significant  difference  in  student 
preference  towards  type  of  video  guide  due  to  the  effects  of 
their  locus-of-control  orientation,  experience,  and/or  the 
interaction  between  locus  of  control  and  experience. 

The  null  hypothesis  was  not  rejected. 
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Hypothesis  13:  There  is  no  significant  difference  in  students' 
expressed  enjoyment  of  the  problem-solving  activity  due  to  the 
effects  of  their  locus-of-control  orientation,  experience, 
perception  of  their  own  problem-solving  ability,  perception  of 
the  problem's  difficulty  and/or  the  delivery  system's  proactive 
help  status. 

The  null  hypothesis  was  not  rejected. 

Hvpothesis  14:  There  is  no  significant  difference  in  students' 
perceived  usefulness  of  video  guide  help  due  to  the  effects  of 
locus-of-control  orientation,  experience,  perception  of  their 
own  problem-solving  ability,  perception  of  the  problem's 
difficulty  and/or  the  delivery  system's  proactive  help  status. 

The  null  hypothesis  was  not  rejected. 

Discussion 

When  sophisticated  computer  systems  are  used  to  augment  human 
performance,  users  are  involved  with  the  task  at  hand  as  well  as  the  tasks 
associated  with  the  artifact  in  front  of  them,  that  is,  the  computer  system 
itself.  According  to  Barnard  (1991),  "To  achieve  impact,  behavioral 
engineering  research  must  itself  directly  support  the  design,  development, 
and  invention  of  artifacts"  (p.  111).  Norman  (1991)  has  written  that  the 
task  artifact  cycle  not  only  draws  upon  and  contextualizes  knowledge  in  a 
particular  domain  but  also  provides  new  knowledge  to  assimilate  to  it. 
Barnard  (1991)  suggests  the  use  of  interaction  scenarios  as  theoretical 
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apparatus  for  studying  and  applying  cognitive  theory  to  the  design  and 
evaluation  of  technology. 

This  study  was  designed  to  examine  user  characteristics  which  may 
play  a  role  in  the  effectiveness  of  digital  video-based  help  systems. 
Specifically,  the  user  attributes  related  to  experience  and  locus  of  control 
were  studied.  Also,  the  way  proactive,  unrequested  help  interacted  with 
these  user  attributes  was  studied.  Finally,  user  perceptions  were 
incorporated  into  models  designed  to  set  the  stage  for  future  research. 

The  study  supported  the  hypothesis  that  increasing  the  amount  of 
digital  video  guide  help  that  novice  users  receive  increases  the  likelihood  of 
success  in  the  problem-solving  activity.  Furthermore,  offering  proactive 
(unrequested)  digital  video  guide  help  increases  the  likelihood  that  users 
will  request  help  on  their  own,  and  users  who  do  request  such  help  are 
more  likely  to  solve  the  problem  correctly. 

It  was  hypothesized  that  both  locus  of  control  and  experience  would 
have  a  significant  effect  on  user  performance  in  a  computerized  problem 
space  equipped  with  a  proactive  help  system.  Several  locus-of-control 
studies  have  concluded  that  internally  oriented  individuals  should  perform 
better  in  problem-solving  tasks  than  externally  oriented  individuals. 
Phares  (1968,  cited  in  Lefcourt,  1976)  found  that  internals  make  better  use 
of  information  than  externals.  Lefcourt  and  Wine  (cited  in  Lefcourt, 
1976)  found  that  internals  were  more  likely  to  attend  to  cues  that  resolve 
uncertainties.  Lefcourt  (1976)  concluded  that  internals  are  "more 


79 


inquisitive,  curious,  and  efficient  processors  of  information  than  externals" 
(p.  65),  while  Horak  and  Horak  (1982)  suggested  that  high  internals  can 
discriminate  general  principles  through  inductive  methods  when  given  a 
guided  discovery  scenario. 

The  results  of  this  study,  however,  indicate  that  locus  of  control  did  not 
contribute  to  achievement  in  the  problem-solving  activity  used  in  this 
study.  The  overall  mean  of  locus-of-control  measures  of  subjects  tested 
was  8.5429  with  a  standard  deviation  of  4.0175,  on  a  scale  from  1  to  40.  A 
lower  index  indicates  a  more  internally  oriented  subject.  The  clearly 
internal  position  of  the  mean  may  have  impacted  the  results.  Similarly, 
experience  had  no  significant  direct  effect  on  performance  in  this  study. 
This  lack  of  a  significance  indicates  that  previous  computer  experience  does 
not  increase  the  likelihood  of  better  performance  in  a  problem-solving 
activity  in  which  the  user  is  inexperienced.  Varying  the  type  of  proactive 
help  had  no  impact  on  the  significance  of  either  the  locus-of-control  or 
previous  experience  effects  related  to  problem-solving  ability.  There  were 
no  significant  interactions  between  varying  help  type  and  either  locus  of 
control  or  experience. 

Although  the  results  of  this  study  indicate  that  locus  of  control  and 
experience  do  not  impact  the  likelihood  of  success  with  the  problem- 
solving  activity,  locus-of-control  orientation  was  found  to  be  a  significant 
predictor  of  whether  the  user  will  find  digital  video  guide  help  useful  and 
of  whether  the  user  will  enjoy  the  problem-solving  activity.  Moreover,  the 
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more  externally  oriented  the  user,  the  more  likely  he/she  will  be  to  (a)  find 
digital  video  guide  help  useful  and  (b)  enjoy  the  activity  itself.  Externally 
oriented  users  also  exhibited  a  stronger  preference  toward  proactive  help. 
Previous  computer  experience  was  found  to  have  no  significant  effect  on 
either  perceived  usefulness  or  enjoyment  of  the  activity.  However,  the 
interaction  between  previous  computer  experience  and  locus  of  control  had 
a  significant  effect  on  both  perceived  usefulness  and  enjoyment.  Moreover, 
less  experienced  users  rated  as  more  external  will  probably  find  digital 
video  guide  assistance  more  useful  and  an  activity  incorporating  such 
assistance  more  enjoyable  than  more  experienced  internally  rated  users. 

Although  the  rather  complex  preference-based  hypotheses  were 
rejected,  several  significant  effects  were  found  in  addition  to  those  related 
to  locus  of  control.  The  study's  findings  indicate  that  users  who  do  not 
consider  themselves  good  problem  solvers  are  more  likely  to  prefer 
proactive,  that  is,  automatic,  rather  than  manually  requested  digital  video 
guide  help.  As  the  problem's  perceived  difficulty  increases,  those  who 
perceive  themselves  to  be  good  problem  solvers  are  more  likely  to  find 
digital  video  guide  assistance  helpful.  Predictably,  users  who  perceive 
themselves  as  good  problem  solvers  are  significantly  more  likely  to  enjoy 
the  problem-solving  activity,  while  users  who  perceive  the  problem  as 
difficult  are  less  likely  to  enjoy  the  activity. 
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Implications 

The  results  of  this  study  indicate  that  digital  video  guide  assistance  can 
be  effective  with  beginners  in  a  problem-solving  activity.  Since  the  use  of 
agents  and  video  guides  is  likely  to  increase,  software  designers  should 
have  an  understanding  of  factors  that  contribute  to  the  successful 
implementation  of  such  digital  characters.  The  variables  in  this  study  had  a 
significant  effect  not  only  on  performance  but  also  on  user  attitudes. 
Interactions  between  variables,  experience,  and  locus  of  control,  for 
example,  also  produced  significant  effects.  Since  locus  of  control  is  a 
construct  used  in  social  learning  theory,  its  application  to  man-machine 
interfaces  is  relatively  unexplored.  Whether  generalizations  concerning 
locus  of  control  extend  to  the  man-machine  interface  needs  further  scrutiny 
(M.  B.  Rowe,  personal  communication,  December  22,  1993).  The  same 
can  be  said  for  other  personality  constructs.  For  example,  educators  and 
software  developers  could  design  similar  studies  which  incorporate 
learning  styles  preference  as  an  independent  variable. 

Several  aspects  of  the  man-machine  dialog  that  merit  further  research 
were  held  constant  in  this  study.  For  example,  the  digital  video  guide 
assisting  every  subject  in  this  study  was  a  white  male  wearing  glasses  and  a 
bow  tie.  As  another  example,  the  proactive  intervention  of  digital  video 
guide  assistance  was  given  on  a  fixed  time  basis  in  this  study.  When 
software  incorporating  help  features  are  designed  and  produced,  an 
understanding  of  how  such  variables  affect  performance  is  extremely 
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important.  In  fact,  Shneiderman  (1991)  believes  that  the  study  of 
performance  variables  provides  a  prime  opportunity  for  conducting 
fundamental  research  in  human-computer  interaction.  Hopefully,  future 
research  will  address  such  issues. 

Recommendations 
The  following  recommendations  are  made  for  future  studies  in  the  area 
of  human-computer  interaction: 

1.  Studies  varying  the  sex,  dress,  race,  speech,  and  gesturing  of  the 
video  guide  could  investigate  learner  preference  on  learning. 

2.  Studies  which  attempt  to  fmd  appropriate  times  to  wait  before 
supplying  machine  intervention  could  parallel  the  wait-time  studies  done  by 
Rowe  (1974,  1986)  in  classroom  settings.  In  a  sophisticated  interactive 
environment  both  wait  time  1  and  wait  time  2  could  be  examined.  Other 
researchers  (Teal  &  Rudnicky,  1992)  have  developed  a  user  model 
composed  of  two  events:  user  planning  and  acquisition  time,  and  execution 
time.  Theorists  attempting  an  analysis  of  the  time  component  of  an 
interaction  scenario  will  find  fascinating  research  on  the  subject. 

3.  The  effectiveness  of  attention-getting  devices,  such  as  gesturing  cues 
used  by  a  digital  video  guide,  could  be  studied. 

4.  The  effectiveness  of  questioning  techniques  could  be  studied  in  an 
interactive  environment  which  monitored  user  action  and  evaluated 
appropriate  behavior  in  a  problem  space. 
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5.  A  study  could  be  designed  to  give  the  subjects  control  over  certain 
attributes  of  their  guide,  such  as  age,  appearance,  voice,  and  clothing.  User 
choices  could  be  mapped  to  learning  styles,  personality  constructs,  and/or 
socioeconomic  factors  to  investigate  if  such  empowerment  aids  the  learning 
process. 

6.  Studies  could  investigate  the  effectiveness  of  incorporating 
proactive  digital  video  guides  in  alert  dialogs  or  specific  computer 
interactions  to  compare  their  effectiveness  with  text  and/or  other 
multimedia  man-computer  dialogs. 

7.  Studies  which  allow  the  user  to  choose  proactive  or  manual  help 
during  a  series  of  activities  could  track  user  preference  from  novice  user  to 
experienced  user.  Is  proactive  help  more  effective  for  beginners?  Does 
proactive  help  enhance  or  interfere  with  the  deliberations  of  experienced 
users? 

8.  Although  often  considered  a  gross  media  comparison,  studies  which 
evaluated  digital  video  guide  assistance  versus  text  only  or  text  and  graphic 
assistance  would  be  useful  to  software  developers. 

9.  Due  to  the  anthropomorphic  quality  of  digital  video  guides,  users 
may  ascribe  attributes  to  guides  other  than  the  ones  intended  by  the  guides 
creators  (Oren,  Salomon,  Kreitman,  &  Don,  1990),  that  is,  external  traits 
of  the  digital  video  guide  may  imply  internal  traits  related  to  value  systems. 
Can  a  digital  video  guide's  implied  expectations  affect  the  learner?  Brenda 
Laurel  (1990)  poses  the  question:  "How  might  interface  agents  affect  the 
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working  styles,  expectations,  productivity,  knowledge,  and  personal  power 
of  those  who  use  them"  (p.  364)?  Studies  which  examine  the  effect  of 
ascribed  attributes  and  its  implications  are  needed. 

10.  Digital  video  guides  and  hand-drawn  cartoon  guides  could  be 
compared  based  on  effective  communication  (i.e.,  instruction)  and  also  on 
ascribed  attributes. 

11.  Based  on  the  number  of  subjects  in  this  study  who  actually  spoke 
to  the  digital  video  guide,  studies  evaluating  digital  video  guides  which 
recognize  speech  versus  those  that  do  not  would  seem  worthwhile. 

12.  Replications  of  this  study  using  subjects  with  a  wider  range  of 
locus-of-control  orientations  could  attend  to  the  null  hypotheses  which 
were  not  rejected  in  this  study. 

Summary 

The  issues  currently  being  examined  in  the  field  of  human  computer 
interaction  are  clearly  in  the  domain  of  educational  research.  Previous 
studies  investigating  teacher- student  interaction  can  be  replicated  using 
interactive  computer  systems  and  artificially  intelligent  software.  The 
ability  of  such  software  to  attend  to  the  wide  variety  of  teaching  and 
learning  styles  offers  fertile  ground  for  instructional  design  research. 

Digital  video  guides  and  agents  represent  a  special  case  of  the 
task-artifact  cycle.  Of  special  interest  are  the  ways  in  which  digital  video 
guides  offer  unique  solutions  to  both  task-related  problems  and  existing 
interface  problems.  For  example,  if  there  is  confusion  during  human 
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driven  searches  of  a  multimedia  environment,  there  is  likely  to  be 
confusion  during  an  agent  driven  search.  Is  it  possible  that,  due  to  the 
anthropomorphic  aspects  of  digital  video  guides,  user  anxiety  can  be 
diminished  when  such  confusion  arises? 

Since  people  learn  and  process  information  in  different  ways, 
providing  help  using  audio  and  video  tracks  allows  users  to  attend  to  the 
information  using  different  resources.  The  addition  of  gesture,  inflection, 
and  other  human  characteristics  to  the  interface  provide  more  tools  for  the 
understanding  and  interpretation  of  the  information  being  presented.  It 
may  be  true  that  these  human  characteristics  can  be  used  effectively  to 
offer  the  user  redundant  input  channels  to  help  clear  up  ambiguities. 
According  to  Laurel  (1990),  "The  metaphor  of  character  successfully 
draws  our  attention  to  just  those  qualities  that  form  the  essential  nature  of 
an  agent:  responsiveness,  competence,  and  accessibility"  (p.  359). 

When  help  is  interactive,  users  can  request  additional  information  and 
perhaps  even  control  its  mode  of  delivery.  If  artificially  intelligent  systems 
are  used,  the  system  can  try  to  isolate  the  source  of  the  problem  and 
proactively  offer  a  solution  to  the  users  of  the  system. 

The  use  of  a  digital  video  guide  represents  a  contextual  forum  for 
virtual  guides  that  assist  in  the  educational  process,  not  the  personification 
of  the  computer,  but  the  virtual  representation  of  a  mediator  in  learning 
activities.  To  what  extent  a  computer  representation  can  augment  the 
learning  process  remains  to  be  seen.  It  does,  however,  seem  that 
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technology  is  attempting  to  formulate  a  response  to  the  problem  that 
Socrates  pointed  out  to  Phaedrus: 

It  is  the  same  with  the  written  words;  they  seem  to  talk  to  you  as  if 
they  were  intelligent,  but  if  you  ask  them  anything  about  what  they 
say,  from  a  desire  to  be  instructed,  they  go  on  telling  you  the  same 
thing  forever.  (Plato,  1961,  as  quoted  in  Norman,  1993,  p.  45) 

This  study  was  designed  to  study  the  impact  of  digital  video  guide  help 
on  users  actively  involved  in  a  problem  space.  Specifically,  it  analyzed  the 
effects  of  several  variables  including  experience,  locus  of  control,  and 
proactive  help  status  on  achievement  in  a  problem  space  and  on  user 
preferences.  The  subjects  in  the  study  were  undergraduate  students 
enrolled  in  an  introductory  instructional  computing  course. 

The  results  of  the  study  indicate  that  increasing  the  amount  of  digital 
video  guide  help  novice  users  in  a  problem  space  receive  increases  the 
likelihood  of  success  in  the  problem- solving  activity.  Providing  proactive 
digital  video  guide  help  also  increased  the  likelihood  that  users  would 
request  help  more  often  which  was  shown  to  be  related  to  achievement. 
Locus-of-control  orientation  was  found  to  be  a  significant  predictor  of 
whether  the  user  will  find  digital  video  guide  help  useful  and  of  whether 
the  user  will  enjoy  the  problem-solving  activity. 
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APPENDIX  B 
COMPUTER  SURVEY 


Computer  Survey  Identiflcation  Code: 


Please  answer  the  following  questions  by  checking  the  appropriate  boxes. 

How  long  have  you  been  working  with  computers? 

 Less  than  6  months   Between  6  months  and  1  year 

 Between  1  and  2  years   More  than  2  years 

How  would  you  characterize  the  frequency  of  your  normal  computer  use? 

 I  use  a  computer  daily   I  use  a  computer  at  least  once  a  week 

 1  use  a  computer  a  few  times  a  semester 

 I  rarely  use  a  computer 


For  what  activities  do  you  normally  use  a  computer?  (Check  all  that  apply) 

 Word  Processing   Drawing,  Painting 

 Electronic  Mail   Spreadsheets,  Budgeting,  Financial 

 Programming   Making  Presentations 

 Research   Work-Related  Activities,  On  the  job 

 Constructing  Databases,  Address  Books,  Appointment  Calendars 

 Entertainment,  Playing  Games 

 Other.  Please  Specify  


Which  of  these  software  packages  have  you  used?  (Check  all  that  apply) 

 Microsoft  Word   Microsoft  Works   Word  Perfect 

 AppleWorks   ClarisWorks   Mac  Write  II 

 PageMaker   Quark   PhotoShop 

 HyperCard   Toolbook   AutoCad 

 Harvard  Graphics   Lotus  1-2-3   Microsoft  Excel 

 Other.  Please  Specify  


Which  of  the  following  Operating  Systems  have  you  encountered? 

 DOS   Macintosh  OS   Apple  DOS,  ProDOS 

 Windows   Unix   Amiga  DOS 

 Other  I  don't  have  any  idea 
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